Secure Large-Scale Outsourced ServicesFounded on Trustworthy Code Executions

Bruno Vavala

Dec 9, 2015

THESIS PROPOSAL

School of Computer Science Faculty of SciencesCarnegie Mellon University University of Lisbon15213 Pittsburgh, PA, U.S. 1749-016, Lisbon, Portugal

Thesis Committee

Prof. Peter Steeenkiste, Co-Chair (Carnegie Mellon University)Prof. Nuno Neves, Co-Chair (University of Lisbon)Prof. Anupam Datta (Carnegie Mellon University)Prof. Antonia Lopes (University of Lisbon)Prof. Vyas Sekar (Carnegie Mellon University)

Submitted in partial fulfillment of the requirementsfor the degree of Doctor of Philosophy.

Copyright c© 2015 Bruno Vavala

AbstractService outsourcing has been particularly incentivized by the Cloud Computing

model. On the one side, clients can benefit from the significantly lower costs forusing computational and network resources. On the other side, however, clientsdo not own the computational infrastructure, which therefore falls out of their trustdomain. Consequently, when somebody other than the client performs the work, itis legitimate to question the correctness and trustworthiness of the service provider.

The fundamental problem is how to enable the service owner, or a client, to getthe benefits of using remote untrusted platforms, while still being able to check thecorrectness of the delivered results. Recently, lots of research effort has focused onfinding solutions that leverage a small and cheap piece of trusted hardware installedinside the remote untrusted platform. However, the secure, efficient and hardware-independent execution of real-world services remains an unmet challenge.

In this thesis we propose practical and general protocols based on a hardwareroot of trust for securing large-scale outsourced services. We target (i) large real-world code bases (e.g., databases, data analytics services) that possibly operate over(ii) a large amount of data. Our objective is building security protocols that delivercorrectness guarantees at a reasonable cost. In addition, we spend some effort tomake our contributions (iii) oblivious to the underlying trusted component, so thatthey can be general and applicable to different security architectures. We proposethree contributions towards this objective.

First, we devise new secure code execution and verification protocols that ad-dress (i) and (iii). The key idea is to separate the concepts of executed code identityand verified code identity. Instead of checking what has been exactly executed, welet the client verify just a small part of the code, which is cryptographically boundthrough our protocols to the remaining code in the correct execution flow of the re-quested service. Such a robust chain enables the client to establish the correctnessof the executed service, while providing the remote server with the flexibility of exe-cuting only the code that is necessary to serve a request. Additionally, our protocolsare based on an abstract trusted component that can retrofit existing ones.

Second, we propose a state management protocol to address (iii). The key idea isusing virtual memory management and efficient authenticated data structures. Thisgives the client integrity guarantees about the (server-side) local state, and it givesthe server the flexibility to load it block-wise and on-demand. We discuss how thetechnique could be applied to (i) and discuss its impact on (iii).

Third, we present the Secure and Verifiable Passive Replication model that lever-ages trustworthy code executions to deliver the security guarantees of Active Repli-cation and the efficiency of Passive Replication. We build a novel replicated systemthat is robust against software attacks, avoids redundant computations, and nativelysupports non-deterministic operations. We discuss how our other contributions im-pact the practicality and generality of such an efficient fault-tolerant system.

iv

Contents

1 Introduction 11.1 Brief Background in TC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11.2 Challenges and Proposed Solutions . . . . . . . . . . . . . . . . . . . . . . . . . 21.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Organization of the Thesis Proposal . . . . . . . . . . . . . . . . . . . . . . . . 8

2 Chained Identities for Efficient Execution and Verification 92.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Trends, Tradeoffs and Challenges . . . . . . . . . . . . . . . . . . . . . . . . . 102.3 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112.4 Completed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.4.1 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132.4.2 Overview. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142.4.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

2.5 Proposed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

3 Dealing with a Large State 193.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193.2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.3 Proposed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3.3.1 Early Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.3.2 Applicability to a Large Code Base . . . . . . . . . . . . . . . . . . . . 23

4 From Active to Trusted Passive Replication 254.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254.2 40 years of replication (almost) . . . . . . . . . . . . . . . . . . . . . . . . . . . 264.3 Finding a Niche to Fit Trusted Computing . . . . . . . . . . . . . . . . . . . . . 274.4 Completed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

v

4.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.4.2 Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 304.4.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.5 Impact of Large Code Execution and Large State . . . . . . . . . . . . . . . . . 314.6 Proposed Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

5 Timeline 33

Bibliography 35

vi

Section 1

Introduction

Cloud Providers, like Amazon [8] and Rackspace [101], support the outsourcing of servicesor the use of supplementary resources so companies can adapt to the dynamic load from theirclients. They can provide flexible on-demand resources, such as virtual machines or disk space,at reasonably low prices. This allows any service owner to reduce the acquisition and operatingcosts of his IT infrastructure. However, as these remote clouds process and deliver dynamic data,it is harder to provide to service owners and clients security guarantees about their outsourcedcomputation, e.g., to verify the correctness of the delivered results.

The problem of checking the results of a remote untrusted computation has been tackledthrough different approaches, and we mention three of them. First, secure computation [52, 137]allows a server to compute any function and to convince the client about the correctness of theresult, by only making cryptographic assumptions. Despite recent significant advances, protocolsfor secure computation remain inefficient and difficult to apply to existing services. Second,replication [71, 73] addresses the problem by making the computation redundant, assuming thatfew replicas can fail, and extracting the correct result out of them. Unfortunately this approach isfundamentally inefficient due to the inherent redundancy. Also, if the replicas belong to a singleuntrusted owner, then the underlying assumptions do not hold. Third, Trusted Computing (TC)approaches the problem by removing the conflict of interest that exists in current deploymentswhere the provider of an outsourced service is also the root of trust for its correctness.

In the following we give some background in TC, discuss some issues and propose how toaddress them. Finally, we provide some related work and delineate a roadmap for this thesisproposal.

1.1 Brief Background in TC

TC is about using a hardware root of trust inside an untrusted platform in order to secure ap-plications and data. Its main objective is to guarantee security properties (such as integrity andconfidentiality) based only on the assumptions that the hardware is correct and untampered andthat its manufacturer is trustworthy. This is essential to remove from the trust domain any pieceof software on a platform (including the Operating System) that a client cannot verify remotely,

1

or that can be compromised, or that simply cannot be trusted.The objective is achieved through two key steps: (i) providing the trusted hardware with

security credentials to (cross-)authenticate itself, possibly through its manufacturer, as a validtrusted device; (ii) letting the hardware make access control decisions and perform operationsusing its own platform state information (such as, for instance, the identity of the running code).

Load andIdentify Execute Attest

Outsource Verify

client(trusted verifier)

server (untrusted third party)

Figure 1.1: Anatomy of a trusted execution. Shaded boxes highlight the contexts of our contributions.

This enables to perform trusted executions as depicted in Fig. 1.1. The client (or a trustedservice owner) outsources the service code and, possibly, also issues some requests. The servicecode is loaded and identified inside the trusted execution environment (i.e., completely isolatedfrom other untrusted applications executing on the platform, including the OS). The identity ofthe code represents part of the platform state information; such information can be augmentedfor example with integrity measurements of the input required by the code and the output iteventually produces. The service is then executed. Upon termination, the identity of the codeand its results (possibly including the received input) are attested using the trusted hardware’scredentials — attestation is a mechanism for securely reporting local platform state informationto a remote party (e.g., the client) — and forwarded to the client. The client eventually verifiesthe correctness of the results and of the executed code and checks that the attestation was issuedby an actual trusted hardware, certified by its manufacturer.

By assuming that a service is outsourced once for serving many requests, a trusted executionis convenient for a client when the verification effort outperforms the execution effort. In practice,however, it is also important to consider the overall effort for the trusted execution (i.e., all thethree steps) at the server side. A higher effort in fact translates into higher latency and cost forthe client, and so less attractiveness.

1.2 Challenges and Proposed Solutions

Finding the challenges. We argue that the challenges lie in the shaded boxes in Fig. 1.1.In fact, service outsourcing is a one-time procedure, which is possibly not performed by theclient itself, but by the service owner/author. Also, the computation steps in a service execution(whether local or outsourced) are ideally platform-independent, and the speed only depends on

2

the processing power of a platform. Finally, the attestation mechanism is typically implementedusing a certified digital signature, which is a typical operation for messages over an insecurechannel. Such operation is independent from the service. However, its efficiency is dependenton the cryptographic algorithm being used (which is out of our scope) and on the platform stateinformation being attested, which is not computed in this phase. This explains why the Attestbox is not highlighted.

The platform state information is security critical and there is some overhead to compute it(in the Load and Identify phase in Fig. 1.1). First of all, it is a function of the current local state(i.e., the software loaded inside the trusted execution environment). It is security critical becauseit can be used by the service during its execution to retrieve security keys and encrypt/decryptdata, and to provide code integrity guarantees through remote attestation. Consequently, suchinformation must be: computed by a trusted entity, protected from tampering, and unique so todistinguish the executed service. Its overhead derives from the function being used (typically ahash function) and the size of the state. It is important to keep the overhead low for two reasons:the information must be available when the security operations occur (e.g., when the servicerequires the decryption of some data), otherwise the overhead can slow down the execution;additionally, the information must be verifiable by the entity that receives it (e.g., a remote client)by making a computational effort ideally lower than executing the service directly. Consequently,such information must be computed fast.

The first challenge is to provide reasonably fresh execution integrity guarantees with lowoverhead at the same time. For example, when the service grows in size (e.g., using a SQL-compliant database engine), if the code loading and identification phase is executed repeatedly,then it becomes a bottleneck due to the service size, which impacts the end-to-end performance;instead, if it is executed only once, then the integrity guarantees eventually lack of freshness.Unfortunately, although hardware and software architectures that enable trusted executions havebeen proposed and analyzed, in general they do not provide freshness guarantees at low overhead.We attribute this issue to the size of the Trusted Computing Base (TCB). Despite significantefforts to reduce it, we believe that current solutions still manage a large TCB.

The second challenge is managing service data efficiently and securely. Any service in factreceives some input data to process, which possibly includes some locally stored state. Since theclient also has to receive integrity guarantees about the data processed by the service, it is desir-able to avoid loading and handling a possibly large input state inside the isolated environmentfor the service execution. In fact, first, the trusted component may not have enough resourcesfor such a large state; second, the service may not even need to access the entire state, but just asmall part of it.

Addressing the challenges above should not come at the expense of the client. New tech-niques should be aimed in fact at making trusted executions more and more cost effective. Hence,a mandatory requirement is to maintain the verification effort for the client constant, indepen-dently from the service and data size and from the execution effort.

An additional challenge is to provide solutions based on a minimal common abstraction ofthe trusted hardware and software security architecture. Unfortunately, given the variety of avail-able TC architectures, an application built on a specific trusted component does not work on a

3

different one, simply because there does not exist an abstraction layer that hides the details of theunderlying component from the application. Hence, an abstraction layer would allow to retrofitexisting trusted components. Additionally, any improvement in security and performance for agiven application (built on the abstraction) would be independent from the trusted component.Proposed approaches. We propose to address the issues above through two main orthogonaltechniques that we devise and use together with a suitable interface to abstract the trusted com-ponent. We evalute their practicality and generality in the popular context of outsourced compu-tation (e.g., Cloud Computing) and of dependable computation (e.g., Byzantine Fault-TolerantReplication).

First, we propose to make the load and identification effort scale with the size of the code thatis required to serve a particular request. We believe this is the key to: further reduce the TCBto the actually executed code; increase security by reducing the time-of-check-time-of-use gap;increase performance since the identification function has complexity linear in the size of thebinary code being identified. For this purpose, we devise a protocol that lets the client securelyverify the correct execution flow of the outsourced service, and allows the server to load and runonly the code modules that are required to respect the correct execution flow.

Second, we propose a new memory management scheme to ameliorate resource consumptionand performance in the case of trusted executions over a large state. The objective is to avoidloading and authenticating the entire state when the execution environment is resource-limited,or when the execution only requires a few state blocks. We tackle this problem by combiningvirtual memory and authenticated data structures.

OSOS

Hardware

TrustVisor / SGX Driver /

Flicker

TrustVisor / SGX Driver /

Flicker

TPM SGX

App AppTCC interface

isolated execution environment

untrusted service proxy

trusted service

trusted service

Figure 1.2: System design and trusted component interface.

In both cases, we strive to devise first a minimal common abstraction for the trusted com-puting component (TCC interface, Fig. 1.2) that can retrofit existing ones (e.g., TrustVisor [85]or Flicker [83] that are TPM-based) or be the reference point for future ones (for instance basedon the upcoming Intel SGX [64]). In the first of our contributions, we find that this is possible,and also get insights about possible optimizations. In the second contribution, instead, memorymanagement can be better handled at a lower level, below the interface.

Lastly, we show the practical benefits of our work in two scenarios. The first is the commoncase of outsourced services, such as database engines and data analytics, running in an untrusted

4

domain. The second is a novel case that we introduce in the Dependability area to improve theavailability of outsourced services through a robust, efficient and simple replicated system. Wepresent the Verifiable Passive Replication scheme. Differently from previous proposals in Byzan-tine Fault Tolerance, the construction enables a single executing replica (i.e., a single executingservice instance) in spite of malicious software attacks and intrusions, thereby reducing the re-dundant computations of a linear factor in the number of faults. The remaining replicas exploitthe efficient and trustworthy verification capability to check the state updates of the executingreplica and so keep themselves up to date securely.

These applications are key to show the importance of our contributions in the context of abigger picture to achieve a larger common goal. Our first technique in fact only targets the codeto be executed. The second one instead precisely targets the data to be processed, though itmay be extended to handle code as data. Each one of them represents a distinct argument ofthe same building block, i.e., the remote trusted execution. Each of our contributions thereforeaddresses an orthogonal issue with the common objective of improving security and efficiencyof outsourced services, which additionally finds applications in secure replicated systems.

1.3 Related Work

There has been extensive research in the Trusted Computing area since the original proposalof a trusted open platform [47, 48] a bit more than a decade ago. The research is oriented atdevising new architectures for a hardware and software root of trust, improving its performanceand increasing its security. Its driving motivation is the creation of an efficient secure architecturethat confines the trust domain to the hardware, the hardware manufacturer and identifiable piecesof code being executed. As a consequence, the trust domain in such model does not include: theOperating System and other services and software running on the platform; the hardware owner(e.g., a user or the platform administrator).

In the following we first present some applications that benefit from such a model and drovethe development of the TC area. Then we review some hardware architectures that representthe root of trust, and some software research prototypes that build on the hardware to enable thetrusted execution of application-level code. Finally, we mention some research efforts for theformal analysis of the correctness of trusted executions.Applications. Trusted Computing techniques finds a number of applications. First, DigitalRights Management (DRM) [102] has been originally a leading motivation for the developmentof the technology. The problem is ensuring that a user can consume some content, without beingable to copy and redistribute it. The solution is to leverage trusted hardware to bind the content tolicensed trusted applications, thereby evicting the user from the set of entities that are trusted inhandling the content data. Another application is enhancing Grid Security [78, 81]. For example,trusted hardware can be used to securely store cryptographic credential [81], so to avoid leavingthem on a platform in the clear; also, it allows to increase the trustworthiness of a platformby attesting that it is running in a known trusted state [78] isolated from external tampering.Other interesting applications can be found in the Cloud Computing area. For example, trustedhardware can be used to attest the current state of a cloud platform to prove its integrity, so that the

5

platform can be trusted to receive and access security sensitive data [22, 107, 108, 109]. Finally,trusted hardware has also been successfully used in in the area of dependability to increase theefficiency of Byzantine Fault-Tolerant State Machine Replication [6, 33, 38, 132, 133].

Besides research applications, trusted hardware is also used in commercial products. Forexample, Microsoft BitLocker [89] uses widely deployed TPMs [124, 127] to protect data inWindows volumes. Amazon CloudHSM [9] provides secure, reliable and durable key storagefor cloud services. IBM offers secure coprocessors [63] on tamperproof PCI boards.

Although trusted hardware is motivated by benign purposes, such as defending from soft-ware attacks, any software on a platform can typically leverage its services. Consequently, evenmalwares can use it, for example to hide their data from analysts [44].

We build practical secure protocols to enlarge the domain of supported applications, so toinclude realistic services such as databases and data analytics applications. The solutions pro-posed in the literature to secure the generic applications that we mentioned unfortunately havelimitations. Consequently, this restricts the domain of specific applications that they can support.Hardware Architectures for TC. Several hardware architectures support TC techniques. TrustedPlatform Modules [124, 127] today represent the hardware root of trust of many commodity sys-tems. They are not general purpose, and only perform cryptographic computations (e.g., hashesand signatures) and store credentials and integrity measurements. They are used jointly withCPUs (with an extended instruction set, e.g., Intel TXT [65]) to set up either a static chain oftrust at boot time [104], or a dynamic chain of trust on demand [55, 56]. The latter is useful toperform trusted executions when they are actually required, by setting up a measured isolatedexecution environment for a small TCB. In the case of Intel TXT, the measured environmentis set up beginning with a CPU instruction (GETSEC[SENTER]) that executes the SINIT Au-thenticated Code Module (ACM) — provided by the manufacturer and included in the TCB —which then transfers control to the loaded and measured code. Some drawbacks of the archi-tecture include: multi-chip usage (i.e., CPU, TPM, memory, etc.), which exposes to physicalsnooping attacks on the connecting buses; direct usage of a low-performance TPM to store themeasurements, which slows down the startup process.

AEGIS [119] proposes a single-chip secure processor architecture that is robust against phys-ical attack by using Physically Unclonable Functions (PUFs), so to avoid non-volatile memory;AEGIS relies on a security kernel that handles memory management, multitasking and authenti-cation mechanisms for secure code executions. However, the security kernel increases the TCB.

OASIS [95] delivers guarantees similar to AEGIS though with a reduced TCB. OASIS is aCPU instruction set extension that provides TC services and limits the security perimeter to theCPU package. However, it requires hardware support.

TrustLite [68] is another hardware security architecture specific for low-cost embedded de-vices. TrustLite enables to set up isolated execution environments for software modules andmutually authenticated channels between them. However, besides requiring hardware support, itsupports a limited number of memory protection regions.

IBM secure coprocessors [63] are designed to be tamperproof and customizable. Though thelatter is desirable to support many applications (see TrustedDB [15]), the former unfortunatelyimposes a high cost.

6

Finally, Intel has recently proposed the SGX instruction set extensions [64, 86] that allow toprotect the memory pages of an application into an Enclave and then to execute it in isolation;interestingly, the trust boundaries are defined by CPU package. Despite the significant progressin novel execution architectures that build upon SGX [17, 111], we believe (and will provideevidence) that the challenges previously discussed are still open.

In this work, we address the challenges by devising protocols above a suitable abstraction ofa trusted component. This allows us to hide the details of the underlying hardware. Hence, wedo not have to target, or optimize for, any specific architecture.Security and Performance of Trusted Executions. Trusted executions aims at providing in-tegrity guarantees about the execution of applications (i.e., they allow a client to verify that thecorrect code has been executed over the correct data). This was informally stated as the LinkingProblem (LP, between code and data) for mobile code executed on untrusted hosts [105, 106].Trusted hardware is an important building block to solve this problem, but to secure applicationssoftware support is useful (if not necessary). We review a few proposed supports to trusted ex-ecutions that use both untrusted software (that cannot mine the integrity of the execution) andtrusted software (that belongs to the TCB and should be ideally free from vulnerabilities).

BIND [112] solves the LP by exploiting trusted hardware based isolation, integrity measure-ments and attestation. Moreover, it highlights the importance of fresh integrity guarantees (i.e.,attested measurements) since the identity of the executed code and data represents the applica-tion’s state only at a certain point in time — later, an attack may compromise the application,whose new identity should then be timely reflected to external verifiers.

The Late Launch technology [55, 56] provides the CPU extensions necessary to set up a dy-namic root of trust (i.e., an isolated and measured execution environment that can be set up at anytime by invoking a special security instruction). Flicker [84] is the first implementation that useslate launch to secure the execution of small pieces of code. Later designs such as TrustVisor [85],Fides [118] and SICE [14] improve respectively performance, ease of application developmentand security by reducing the trusted computing base.

Finally, Haven [17] leverages Intel SGX technology to protect the integrity and confidential-ity of code executions; also, it leverages a library OS [100], included in the trusted executionenvironment, to protect applications from Iago attacks [31] and to support the executions ofunmodified applications.

Wrapping up using Fig. 1.2, these designs can be roughly considered middleware for securedapplications. They sit on top of the trusted hardware and they have some untrusted servicerunning on the untrusted OS. This is useful for example to forward data to/from the network;in fact, since the data is usually secured at the application layer, it is reasonable to leave theimplementation of lower layers out of the TCB.

Our objective is to devise secure and efficient protocols that are possibly applicable to avariety of such software architectures. Our interface to a generic trusted component aims atexposing the common capabilities (such as code execution, secure storage, attestation) that eacharchitecture offers. Our new identification scheme for actively executed code is entirely builtabove such interface. We believe however that lower level mechanisms (below our interface, forinstance using virtual memory) are more suitable to deal with a large state.

7

Formal analysis of trusted executions. There has been research oriented at modeling, formal-izing and proving security properties for secure systems and protocols. In [5] a logic is used forthe analysis of NGSCB — Microsoft’s former proposal for a secure computing base. The logic isreminiscent of the BAN logic [27, 28], and subsequent sound semantics [4] and extensions [120],used to reason about authentication protocols and belief propagation. A logic of secure systemsis proposed in [41] and used to analyze trusted executions based on late launch. More recently,Moat [113] has been introduced to formally verify confidentiality properties of applications thatuse Intel SGX.

In our proposed work, we argue that a logic for reasoning about belief propagation amongexplicitly trusted software principals can help the formal analysis of our trusted execution proto-cols, so to prove their correctness. Unfortunately, it is still not clear whether an existing formaltool can be immediately applied or should be adapted first.

1.4 Organization of the Thesis Proposal

Section 2 In this section we design implement and evaluate a new code execution protocol thatis secure and low-overhead and can retrofit some existing trusted components. The key idea isthat we refine the definition of code identity to treat and guarantee separately execution integrityon the untrusted platform and execution verifiability by the client; then we use a cryptographicmechanism to ensure that the latter implies the former. Interestingly, our new construction canretrofit existing trusted components and suggests important optimizations. As future work, wepropose to prove formally the correctness of our protocol.Section 3 After dealing with large code bases (in the previous section), in this section we tacklethe problem of how to manage a large amount of data required for the execution of a large-scale service and how to preserve its integrity. We motivate the work and present a preliminarysystem design to handle a large state efficiently. The basic idea is to leverage virtual memorymanagement jointly with efficient authenticated data structures.Section 4 In this section we present a novel application where our techniques are useful, whichfurther motivates our work, and serves as the basis to improve the availability of secure out-sourced services. We use trusted executions in the context of fault-tolerant replication to buildmore efficient replicated services. Our contribution is a new arbitrary-fault-tolerant replicatedsystem that only requires one executing replica. Such a construction greatly departs from previ-ous Byzantine Fault-Tolerant replication techniques that require lots of computational resources,deterministic services and expensive coordination protocols. As future work, we propose a for-mal analysis of our replicated system.Section 5 Finally, we present a tentative timeline to complete the work proposed in previoussections, including additional research work, prototype implementations, paper submissions, aswell as preparation and defense of the thesis document.

8

Section 2

Chained Identities for Efficient Executionand Verification

2.1 Background

Let us briefly recall the anatomy of a trusted execution (Fig. 1.1). The outsourced service codeis loaded and identified in the isolated and untampered environment set up by the trusted compo-nent; it is then executed, attested and eventually verified by the client. The chain that allows theclient to trust the remote execution is composed by: the hardware manufacturer (i.e., the root oftrust) who vouches for the trusted component who vouches for the service code execution.

manufacturer trustedcomponent

servicecode

private key

public key

private key

public key

0 1 0 1 0 1 0 1 1 01 0 0 1 1 0 0 1 0 1. . . . . . . . . .. . . . . . . . . .. . . . . . . . . .0 0 1 1 1 0 0 1 1 11 0 0 0 0 0 1 0 1 1

H(code)

root certificate installed at the client

certificate validation

certification

attestation

attestation verification

code identification

Figure 2.1: Chain of trust from the manufacturer to the executed service code.

The client verifies the chain as depicted in Fig. 2.1. It is assumed that the client knows (orcan securely retrieve) the root certificate that belongs to the trusted hardware manufacturer and,additionally, the service identity — typically defined as the cryptographic hash of the servicebinary. The client uses the certificate assigned to the trusted component to verify that the at-testation covers the known service identity. Then, the manufacturer’s root certificate is used tovalidate the trusted component’s certificate.

Code identity is therefore a security-critical ring of the chain, and a key concept in TC.It serves three main purposes: it informs a client about what has been executed; it providesservice code integrity guarantees, since it is defined as a cryptographic hash; it allows the trusted

9

component to implement access control mechanisms for secure operations (e.g., key retrieval,secure storage) on a per-service-code basis.

2.2 Trends, Tradeoffs and Challenges

Trends. As the technology improves, adopting Trusted Computing techniques to secure appli-cations is becoming more attractive and cost-effective. The hardware is getting cheaper [124],minimal [95] and faster [64]. Larger and richer applications can be deployed — e.g., TrustedDB[15, 16] secures an SQL database engine on an IBM 4764 cryptographic coprocessor. Supportcan be provided even for unmodified binaries [17]. Consequently, the technique appears to be aviable alternative to secure computation [96], which is more secure but also significantly moreexpensive.

10

30

50

70

0 500 1000 1500 2000

Late

ncy

(ms)

Code (KB)

protection overhead (avg)

Figure 2.2: Security-sensitive code registration latency in XMHF-TrustVisor [85, 130]. It shows a linear dependencebetween code size and protection overhead.

Effects. As richer and larger applications are deployed, the cost of code identification becomesnon-negligible. As an example, we measured this cost in XMHF-TrustVisor1 (Fig. 2.2). Thefigure shows the time to register different code bases. During this procedure, the memory pagesof the code are isolated and identified. The registration time is linearly dependent with thecode size, reaching about 37ms for only 1MB of code. Such linear dependence holds also forIntel SGX [64, 86], used to build secure Enclaves. In fact, after an Enclave is created andprotected (ECREATE instruction), code pages must be added and measured (EDD, EEXTENDinstructions), before finalizing the Enclave and fix its identity (EINIT instruction). Therefore,the overhead of creating an Enclave identity grows with the code size.Tradeoffs. Trends and effects allow to prefigure an important tradeoff between security and effi-ciency in trusted code executions. We spot it by reviewing the two alternatives that are currently

1XMHF-TrustVisor [85, 130] is a hypervisor-based trusted computing component designed to be secure andefficient. In our configuration it leverages Intel TXT [65] and a TPM v1.2 [127] as a hardware root of trust tosecurely bootstrap itself. The hypervisor is designed to provide TPM-like services, and includes mechanisms for theefficient measured and isolated execution of small pieces of code.

10

available to deal with large code bases.We dub the first as measure-once-execute-forever [17]. The integrity measurement (or code

identity) is established immediately before the execution. The execution can then proceed in thetrusted environment indefinitely. The approach is secure and amortizes the code identificationcost throughout the execution. However, the integrity guarantees about the code only holds at theprecise point in time when the measurement is taken. Therefore, later successful attacks can goundetected. This problem is also known as time-of-check-time-of-use (TOCTOU) discrepancy[112].

The second alternative is dubbed measure-once-execute-once. The integrity measurementis repeatedly established before each execution (e.g., a Flicker [83] based application). Thisapproach is secure. Additionally, it provides fresher integrity guarantees. However, it does notamortize the identification cost, so it raises efficiency issues.Challenges. The main concerns in securing large applications are that (i) the time intervalbetween consecutive integrity measurement can get high and (ii) the identification cost growslinearly with the size of the code that may be executed, not the size of the code modules that areactually executed. When the code base is large and the actually executed code is only a smallfraction, then (ii) implies that the identification cost can have a significant impact on the overallexecution cost.

The challenge is therefore addressing the concerns above so to increase security and effi-ciency of the actually executed code. Interesting solutions should come at no cost for the client,i.e., the verification effort of an application should remain as efficient as in current techniques.Additionally, solutions should require only software contributions, so to retrofit existing trustedcomponents.

2.3 Related Work

Code identity and trusted executions. Code identity has been originally defined as the digestof a program’s code in [74]. The same definition was later borrowed for trusted hardware-basedcode executions [47, 48] as a useful mechanism for sealed storage and attestation purposes.Current platforms and CPU extensions such as AEGIS[119], Intel TXT [65], Intel SGX [64],OASIS [95], TrustLite [68] allow to identify some code before the execution by hashing itscontent. Tools that leverage some of these architectures, such as Flicker [83], TrustVisor [85],Haven [17] do not address the problem of code size inside the trusted environment and executemonolithic applications, whose identity can be verified remotely. In our work, we do not changethe definition of code identity (i.e., the hash of the binary), and we observe that another way forthe client to verify a remote execution is to (be able to) make trust inferences. Therefore, bybuilding a robust chain of trust throughout the modules of a large code base, it is sufficient forthe client to verify only part of the chain to infer that the execution of the whole code base wasperformed correctly.

OASIS [95] proposes to deal with an application whose size is greater than the cache bybuilding a Merkle tree over its code blocks. However, it requires new hardware support and

11

it does not provide general solution that retrofits existing trusted computing components. Ourproposed protocol instead could leverage OASIS by implementing our TCC abstraction and,with minimal modifications, it could also include our novel secure storage construction.

The BIND service [112] leverages fine-grained code attestation to secure a distributed system.However, BIND targets small pieces of code, while our protocol is able to provide executionintegrity guarantees of large code bases. Additionally, although small modules could use BINDto build a chain by verifying each other, the resulting construction would not be verificationefficient and could incur verification loop issues. Our protocol solves these issues and guaranteescontrol flow integrity when the client eventually verifies the last executed module and the identitytable (later defined). Finally, BIND’s security kernel was not implemented [83].

Another mechanism [32] proposes an efficient verification primitive in which some code forhash computation is embedded in the software code at compile-time. The program executionis traced by deterministically hashing inputs and outputs of sequences of instructions. Unfortu-nately, it requires some re-computation, which diminishes the advantages of service outsourcing.Also, its security relies on the strong assumption that the adversary does not perform code anal-ysis.

A research work related to ours is the On-board Credential (ObC) Project [26, 46, 69]. It isan open architecture based on secure hardware [125, 126] for the installation and execution ofcredential mechanisms on constrained ObC-ready (typically mobile) devices. It enables a serviceprovider to provision secrets to a family of (installed) credential programs [69], which are exe-cuted slice-wise in a secure environment [46], possibly using the TPM’s late launch mechanism[26]. The architecture focuses on secure execution of credential programs that are applicationor platform-specific, while our work is concerned about the efficient verification of executionsthat are performed by means of a generic trusted component. The chain of trust among the slicesis based on the slice endorsement token, containing the family and program-specific secrets,which is created online on a per-slice basis. In our case, the chain is made explicit in each PALthrough a reference to the identity of the previous/next PAL, and only needs an offline setup (i.e.,the process of making the code base available on the UTP) performed by the service authors.Also, access to secure stored information is controlled by (our) construction through the trustedcomponent, allowing secure data exchange among PALs pairwise.

2.4 Completed Work

We outline a protocol for code execution and verification that breaks the coupling between ser-vice code size and cost of identification. The protocol has two main desirable features. First,the trusted architecture loads, identifies and runs only modules of the code base that are actuallyexecuted. This provides execution flexibility on the untrusted third-party platform (UTP) andsaves resources on the trusted computing component (TCC) being used. Second, the protocolguarantees the correct execution sequence of code modules by means of a robust and verifiableexecution chain. Each module secures the application state through a secret key that depends onits own identity and on the identity of the next module in the correct execution flow. These twomechanisms combined enable a secure and efficient identification.

12

The protocol further enables efficient verification on the client side. In fact, the client checksthat the correct code was executed by only verifying a chain endpoint to bootstrap trust in thewhole chain. This has two significant benefits: first, code modules that are not executed do nothave to be loaded or verified; second, the client does not need to be aware a priori of the exactexecution order.

Summarizing, we aim at making the code identification cost scale with the size of the actu-ally executed code, instead of the size of the code base as a whole. This can ultimately balancethe cost of re-identifying some code for freshness guarantees. We achieve the following goals.1) Secure proof of execution: the proof of execution of the correct code is unforgeable, unam-biguous, and linked to the hardware root of trust. 2) Low TCC resource usage: the protocol issecure with minimal resource (code identification, cryptography, storage, etc.) demand on theTCC. 3) Verification efficiency: the overhead for the client is constant, independently from thecode base—i.e., a fixed number of hashes and digital signatures. 4) Communication efficiency:the protocol is ”non-interactive”, requiring only a small additional constant amount of traffic toenable successful client verification. 5) TCC obliviousness: the protocol is ”oblivious” to thespecific TC architecture actually used, thus allowing to retrofit existing trusted components.

2.4.1 Model

System model. We consider an untrusted third party (UTP) platform, such as a remote cloud,equipped with a trusted computing component (TCC). An adversary may take control of anysoftware running on the UTP machines, including services and the operating system, therebybeing able to read and modify any data. The adversary cannot perform physical attacks but isallowed to use the TCC through its interface, for instance executing tampered code modules. ATCC can be implemented using current TPMs, upcoming Intel SGX [64] or secure coprocessors[115].Code base. Our code base is composed by m modules (or PALs2) A correct generic executionflow is represented by a finite sequence of possibly repeating PALs executed in the correct order.UTP-side. The code base is available on the UTP, possibly outsourced by its (trusted) authors.Client-side. The client knows the cryptographic hashes (160 bits each, using SHA1) of theattested PALs. Also, the client knows the hash of the PALs identity set — i.e., the set of identitiesthat constitute the code base. Ideally, the information could be provided by the (trusted) authorsof the code base and it requires a constant amount of space.

Additionally, the client knows and trusts the TCC’s public key. This can be achieved throughan initial TCC Verification Phase: the client interacts with the UTP to retrieve the TCC’s publickey and its associated certificate; if the public key is correctly certified by a trusted CertificationAuthority, then it can be trusted and used to verify remote executions.

13

A()!

B()!

ID(A) A34D39FB… ID(B) B71686EC… ID(C) E323AFEC…

code base CB

Hardware!

OS!

Apps!CB!

trusted!component ! !!secret sharing

C()!

identity table

secure!!!channel!

1.!

2.!

3.!

4.!

Figure 2.3: Sketch of our solution.

2.4.2 Overview.

The core of our solution is displayed in Fig. 2.3. On the left, the code base is depicted as a set oflogically connected modules (arrows express the control flow graph) stored on the UTP, workingtogether to provide a service. On the right, our protocol works as follows. It executes in sequenceon the TCC (1) the modules of the code base that are necessary for the requested service (e.g.,C is not executed and not loaded). Given a particular client request, only the modules requiredto serve it are considered active (A and B in the figure). Active modules are loaded and runaccording to the correct execution order. Each module secures (2) the intermediate state before itterminates. The next active module is then executed (3) and it validates the previous intermediatestate (4). Such state is passed through modules by means of logical secure channels.

Each module has its own identity. This is in contrast with having a single identity assignedto the code base. Instead, we associate the code base with the set of the identities of its modules(i.e., the identity table), while our protocol guarantees that their execution order respects thecorrect control flow graph. Additionally, an identity is still defined as the hash of the code. Thisallows to maintain backward compatibility, so to achieve a general solution that is implementableon current TCCs.

Each executing active module has access to data and resources required for the computation.Before the execution, each module is expected to receive some input from the client (e.g., arequest) or some intermediate state from other modules. Similarly, when it terminates, eachmodule is expected to produce either an output for the client (e.g., a reply) or some intermediatestate to be processed by the next module in the execution flow. Before and after the execution,every piece of data to/from any module is handled by the UTP in the untrusted environment;consequently, it must be secured by means of the available TCC resources (e.g., secure storage).

An executing active module has access to the identity set of the code base. This allows anactive module to leverage TCC-based access control mechanisms to secure data. Intermediatestates are securely transferred between modules through logical TCC-backed identity-dependentsecure channels, whose security is enforced by the TCC. Such channels are ”logical” in that theprotected data is transferred through modules by the UTP (i.e., A releases data to the UTP and

2Piece of Application Logic. This maintains the notation consistent with previous works [83, 85].

14

it is unloaded, then B is loaded and receives data from the UTP). The channel is secure as itguarantees data integrity, through message authentication codes, and the authentication of theend points based on their execution order (i.e., A sends to B and B receives from A, but B andC will never exchange data). An important advantage of these channels is that they allow tomaintain all the information locally (UTP side) thereby avoiding the interaction with the clientduring the execution.

A client reply is authenticated through a TCC attestation, or through previously establishedsymmetric secret keys. The attestation eventually received and verified by the client only coversthe identity of a single module. In our case, the last executed module (in the control flow graph)calls the TCC attestation service, and the TCC attests the module’s identity. Such last modulecan however include (the integrity measurement of) some parameters in its attestation, such asthe client’s initial request, the identity set of the code base, and the client’s reply. The attestation,together with the parameters used to generate it, represents a proof of execution that can bechecked by the client. By verifying the module’s identity and the identity set, the client can trustthat the code base correctly served the request.

Noticeably, the proof does not reveal what modules were actually executed (besides the at-tested one). Consequently, the client does not need to verify each executed module, therebymaking the verification process efficient. Also, the client does not need to be aware of the exactexecution sequence of the code modules, which is a desirable feature.

2.4.3 Evaluation

PAL0

PALINSPALSELPALSQL

PALDEL

Client

Tabid(PAL0)

id(PALSEL)

id(PALINS)

id(PALDEL)

id(PALSQL)

selectquery

insertquery

deletequery

anyoth.query

request reply

Figure 2.4: Design and example execution of our trusted SQLite database engine.

Trusted SQLite Design. We apply our protocol to the SQLite database engine [117]. SQLitehas a non-trivial large code base of about 88K lines of source code. We chose SQLite because itis open-source, downloadable and widely deployed for instance on Android [1], iCloud [3] andother operating systems [2]. We implemented our TCC through XMHF-TrustVisor.

We created 5 PALs organized as shown in Fig. 2.4. PAL0 is the first one called from theuntrusted environment on the UTP and receives input data from the client; it only performs veryfew operations to parse the client’s request to recognize the type of query. The request is thenforwarded to the PAL that is responsible for the execution by means of our secure channels based

15

on identity-dependent keys. As the names suggest, select queries are passed to PALSEL; insertqueries are passed to PALINS; delete queries are passed to PALDEL; any other query is passedto PALSQL. The first three PALs are each specialized to perform a specific query (i.e., theyonly implement part of SQLite) The last PAL instead can execute any query (i.e., it implementsthe full SQLite engine); it has been implemented to make our multi-pal SQLite fully functional.The last executed PAL then builds the reply that is eventually released to the UTP’s untrustedenvironment and forwarded to the client.Results. We analyzed the size of the code for each PAL, as protected by XMHF-TrustVisorat registration time. The size of the full SQLite implementation is about 1MB, while commonoperations such as select, insert, delete can be implemented in as little as 9-15% of the codebase.

Overall, our protocol improves substantially on the baseline approach, represented by theexecution of the monolithic SQLite database engine. For example, for insert, it is about 1.46×faster than the traditional approach using the monolithic SQLite. The result could be improved tobecome up to 2× faster by considering faster attestation mechanisms. We conclude that our pro-tocol can improve performance, enable repeated attestations to provide fresh integrity guaranteesat a reasonable cost for a real-world application and, additionally, it can reduce the time-of-check-time-of-use gap.

2.5 Proposed Work

Proposed work. An important piece of work that could strengthen significantly our securitymosaic is to prove the correctness of our general protocol. In fact, although one execution of ourprotocol in Fig. 2.4 has been verified, the correctness of our general technique remains unfortu-nately unproven.

We have already made some progress in this direction. In particular, we have extended theBAN logic [27, 28] to make trust explicit. We have modeled our protocol in the logic. We havebeen able to argue the correctness of our protocol. In particular, we have shown that using alogic of authentication to reason with such protocols is simple; the logic enables a more carefulanalysis of the security properties and of belief propagation3; it is easy to extend the logic toexpress explicitly the set of trusted principals whose statements can be accepted, and whichincludes service applications besides hosts and platforms.

Unfortunately, the BAN logic is known to be unsound [23]. A sound semantics was latergiven in [4], clarifying several aspects of the logic (e.g., belief propagation, honesty, good keys,etc.). An extended sound logic that unifies previous proposals was later devised in [120]. Thelogic of secure systems in [41] can be used to analyze late-launch based executions. For ourwork, we are looking for a formal tool that can provide a simple and sound framework to reason

3Beliefs about true statements are transferred from/to trusted principals by means of messages sent throughinsecure channels or forwarded by other untrusted principals. A secure protocol like ours essentially provides thesecurity chain that prevents any malicious activity from tampering with the statements generated by, and deliveredto, trusted principals.

16

about authentication protocols where trust can be expressed explicitly4.Backup plan. There are some risks in devising the formal tool above. In the case the goalcannot be reasonably achieved, our backup plan is to focus on the work proposed in Section 3.3.

4In [4] it was already recognized that trust (or honesty) of the involved principals is not a reasonable assumption.

17

18

Section 3

Dealing with a Large State

In this section we provide some background about the importance of security applications thatoperate over a large state. Also, we review some related works in the area that follow differentapproaches. Finally, we propose our approach to the problem and devise a preliminary designfor a large state management architecture applied to a trusted computing component.

3.1 Background

Offloading computationally-intensive tasks (e.g., data analytics, or database outsourcing) to thirdparties is more and more customary. Cloud computing for example has increased the flexibilityand lowered the cost for renting computational resources, thereby incentivizing computationoutsourcing.

Securing remote applications is an active area of research. There is a twofold reason. First,third-party platforms typically do not belong to the trust domain of the entity that outsourcescode and data. Second, outsourcing security-sensitive information (e.g., health care or financialdata) requires protocols and tools that enables customers to check the remote activity.

As these applications get richer and therefore larger, there are new challenges to overcome.We refer specifically to applications requiring a large state, such as databases or data analyticsin genomics [37, 57], ranging from megabytes to hundreds of gigabytes. Computing at sucha large scale requires efficient protocols for resource and data management. Adding securityrequirements significantly raises the bar. First, new protocols are necessary to maintain local (onthe remote platform) integrity guarantees, while ensuring efficient resource utilization. Second,a mechanism is necessary to convey the local security properties to the remote client that wantsto check them. Third, the new security architecture should incur a reasonably low performancepenalty.Recent advances. There have been recently a few research efforts that aimed at increasingthe security of real-world applications such as databases and large-scale analytics. TrustedDB[15] leverages a cryptographic processor to protect the confidentiality of a SQL database and toenable private data processing. CryptDB [99] provides confidentiality to DBMS-based applica-

19

tions, without requiring extra trusted hardware. CryptDB executes SQL queries over encrypteddata handled by unmodified DBMSs, and avoids the performance penalty of fully homomorphicencryption. Finally, VC3 [111] allows to run MapReduce computations securely using Intel SGX[65]. Interestingly, VC3 leaves the operating system and Hadoop itself out of the TCB, thereforeminimizing the code to be secured.

3.2 Related Work

We review some related works in the area that follow different approaches to deliver integrityguarantees to applications that operate on large state.Memory Checkers. The research area arguably began with the notion of memory checkers(MC) [21]. MC have a small reliable memory that can be used to validate data coming from alarger untrusted source. [21] studies four types of checkers; in particular, it presents an efficientonline (i.e., operation-by-operation) checker based on Merkle trees with a O(t log n) overhead(with t the time to compute a pseudo-random function, and n the memory size). Subsequentworks [45, 92] improve the efficiency of checkers and show space-time tradeoffs to solve theproblem in the information-theoretic and computationally secure settings.

These results apply to our setting. A trusted execution environment typically has limitedresources and, anyway, it is typically not designed to handle a large amount of data as a whole.Therefore, in order to leverage an external untrusted storage service, a trusted service has toimplement mechanisms to check and secure the integrity of non-local data.Authenticated data structures. Authenticated data structures have been proposed to check andupdate efficiently small blocks that belong to a large database. In order to avoid to load andauthenticate the entire database, a data structure for single block authentication is built over thedatabase. In [18] incremental MACs (message authentication codes) have been introduced toauthenticate and modify efficiently large messages. The construction however assumes that themessage is entirely stored in trusted memory. Another construction that drops this assumption isbased on Merkle trees. It has been later shown [49] that incremental cryptography is related tomemory checkers.

[20, 60] provide data structures based on incremental cryptography and authentication treesthat are highly parallelizable. Their complexity however remains logarithmic in the number ofdata blocks.

A study of the cost of authentication using hierarchical cryptographic hashing is performedin [121, 122]. They focus on the dictionary problem (i.e., answering membership queries) andprove a logarithmic lower bound to solve the problem, which makes tree data structures [87]optimal.

Another alternative to solve such problem is the use of cryptographic accumulators [29, 54,77]. These are typically based on asymmetric cryptography (e.g., using RSA) and are consideredless efficient in practice. However, interestingly, they can achieve constant verification time forsingle block authentication.

We build on authenticated data structures for time and space efficiency. They are particularly

20

indicated in constrained environments, and fast (if based on symmetric cryptography). Also, theyscale well with the size of the protected data.Caches. [34, 51, 119] analyze hardware solutions for fast memory integrity verification. Theirsolutions are based on Merkle tree for the authentication, but use on-processor caches to speedup the computation of the hash tree.File Systems and remote storage. Other works use constructions based on Merkle trees tosecure filesystems [94] or remote cloud storage [98, 123, 136]. [94] achieves constant spacecomplexity, and it leverages the characteristics of typical file-system workloads, such as lowentropy (e.g., in text files) and sequential writes. [98, 123, 136] directly tackle the problem ofensuring data freshness for remote put/get operations. CloudProof [98] however uses Merkletrees only when the cloud is periodically audited by the data owner; between audits, it relies onthe history of cloud signatures to detect any misbehavior.

In our setting, a filesystem is currently unavailable for a trusted execution. We propose tobuild an in-memory mediator for read/write file access. This allows to expose a simple interfaceto trusted services. Also, it allows to keep the TCB small by leaving the filesystem manage-ment to the untrusted OS, and implementing only file authentication operations inside the trustedenvironment.

3.3 Proposed Work

Objective. We propose to design, implement and evaluate new security protocols and mecha-nism for general and efficient trusted executions over a large state. First, we want to address theproblem for trusted-hardware based executions. The technology is in fact making progress, andthe upcoming Intel SGX appears to be promising in delivering reasonable performance for real-world applications — arguably higher than solutions purely based on cryptographic schemes.Second, we want to benefit from the latest trusted component, without forgetting the previousones. Hence, solutions should find a common abstraction between trusted components, specify-ing an adaptation layer to use any of them. Third, though we target database engines, we wantto be able to support a larger spectrum of applications, for example genome sequencing [76] andanalytics over sensitive data [37].How related works fall short. The previously mentioned recent research efforts (Section 3.1)propose ad-hoc solutions to important problems. TrustedDB [15] is both application-specificand trusted-component-specific; besides, it has an elevated cost due to the secure tamperproofcoprocessor. CryptDB [99] is also application-specific and it does not guarantee integrity. VC3[111] provides a general solution for MapReduce applications, but it is tightly built on IntelSGX. Finally, XMHF-TrustVisor [85, 130] is a hypervisor-based trusted component to securelyrun pieces of code in isolation. However, as the input size increases (see the number of memorypages in Fig. 3.1), the overhead for protecting it increases linearly with its size.

21

10

30

50

70

100 300 500 700 900La

tenc

y (m

s)

Memory Pages

(max) protection overhead(min) --(avg) --

Figure 3.1: Security-sensitive memory protection overhead in XMHF-TrustVisor [85, 130]. It shows a linear depen-dence between memory size and protection overhead.

3.3.1 Early Design

Our early design is based on three concepts of: protecting the initial state offline, so to make ittamper-evident; registering its authentication information in the trusted environment, so that thestate can be later verified by a remote client; lazy loading of state blocks, so to securely improveresource utilization in a trusted execution. These concepts have to be further investigated to un-derstand their feasibility on different security architectures and to come up with a general design.Then, we will proceed with their implementation and evaluation over large-scale database anddata analytics applications.Offline integrity protection. The state is integrity protected using an efficient authenticateddata structure. For simplicity, we assume that this is performed on the trusted host that origi-nally stores the state. The data structure adds complexity to the data in terms of authenticationinformation, but it also allows secure and efficient handling of small parts of the whole state.Verifiable state registration. The idea is to register (or initialize) both the state and the au-thentication information for the trusted execution before running it. The objective is building amechanism that does not need loading the whole state upfront immediately before the executionbegins. On the one hand, the mechanism should allow to keep the large portion of the state storedin the untrusted part of the platform. On the other hand, it should guarantee security in terms ofintegrity and freshness of the whole state.Lazy authenticated loading. The state should be loaded on-demand, as required by the ap-plication, which is also common practice in today’s operating systems. The idea is to leveragevirtual memory mechanisms and page fault interrupts to make the whole process transparent tothe application.Generality of the approach. We believe the approach is feasible for architectures that can beeasily customizable, such as XMHF-TrustVisor. Its feasibility of the newer Intel SGX has to bestudied.

22

3.3.2 Applicability to a Large Code Base

Since the binary code of a service is essentially a sequence of bits that can be considered part ofthe service state itself, then an immediate question is whether the proposed approach is applicableto the service code as well. Since this requires on-demand loading of the code to be executed,we need some means to ensure that the code belongs to the large code base.

This is an orthogonal issue that can be solved using the knowledge acquired in Section 2.The design can therefore be extended appropriately. There are two crucial observations: i) theconcept of PAL that we used to deal with large code bases (in Section 2) is generic; ii) the identitytable is not executable code but rather data. Hence, our design could be extended by leveraging(i) to define PALs as code blocks. This implies that the identity table contains the identity ofeach code block. Additionally, since the blocks may be fine-grained, thereby resulting possiblyin a large identity table, we can leverage (ii) to maintain the table as part of the service data, soto load and use it efficiently as well through the basic design outlined above (Section 3.3.1).

23

24

Section 4

From Active to Trusted Passive Replication

We present a novel Trusted Computing application that leverages hardware-based security toexploit the power of efficient computation verification in fault-tolerant replication. This con-tribution allows to improve the availability of secure outsourced services. We show how ourprevious contributions and the works proposed in previous sections impact on this application.

4.1 Background

Over the past decades, several papers have been published in the area of Fault-Tolerant Replica-tion. These solutions ensure service availability by resorting to component replicas, which cankeep the system operating correctly in case a subset of them fails. In turn, this approach raiseschallenges related to the management of component replicas. There are typically two approachesto address such challenges: active replication (AR) [71, 72, 110] and passive replication (PR)[7, 25, 93]. In the research line that considers arbitrary failures (also known as Byzantine) [73],to the best of our knowledge, all past solutions are based on the former, commonly referred to asthe State Machine Replication (SMR) paradigm [71, 110].

In the following we review the AR and PR designs and we analyze their robustness againstmalicious failures.Active Replication. In AR, a significant fraction of the replicas run service operations (PBFT[30] is a well-known example). Besides the overhead of executing such redundant operations,expensive and complex coordination protocols (e.g., consensus or atomic broadcast) have to befollowed in order to maintain consistency in a distributed system. Although significant efforthas been spent to make Byzantine Fault Tolerant (BFT) SMR practical [35, 38, 50, 58, 66, 133,135], these inherent issues persist. Additionally, BFT-SMR has to deal with non-deterministicexecutions that may cause the distributed system state to diverge [11, 67, 114, 129]. The solutionis often to assume that the service is deterministic, thereby restricting its applications.Passive Replication. In PR instead, only the primary replica runs the service, while the backupreplicas receive and apply state updates. The technique does not tolerate arbitrary failures be-cause a malicious primary could compromise the system state undetectably. However, there are

25

many practical examples that evidence the attractiveness of the approach in a crash-only scenario[53, 62, 79, 80]. The centralized control offered at the primary is in fact resource-efficient andenables an easy and cheap coordination of the other replicas. In addition, since only the primaryhas to process the requests, non-deterministic executions are allowed.AR vs. PR. Both BFT-SMR and PR have advantages and drawbacks. Their schemes are typi-cally characterized by middleware’s, placed right below the service, that guarantee correct repli-cation. On the positive side, middleware and service are independent and unaware of each other’sinternals, making the approach fully general. On the negative side, even corrupted software (pos-sibly due to an intrusion) is allowed to run above the middleware. Despite this, correctness inBFT-SMR is attained by fully replicating the service (i.e., the state machine), and assumingthat enough replicas are not compromised. Instead, PR cannot achieve correctness because theprimary could become erroneous and its execution is not replicated.

4.2 40 years of replication (almost)

The concept of arbitrary fault tolerance dates back to the seventies. In the SIFT Project [134],in order to implement a reliable system for aircraft control, it was critical to cope with arbitraryfailures of its components. The system was organized in a topology of interconnected replicatedprocessors, each one having exclusive write access to its private memory. An algorithm forclock synchronization was originally devised to tolerate f simultaneous failures, out of 3f + 1available clocks. The fundamental problem was later formalized [97] and such failures becamebetter known as Byzantine failures [73].

In the past two decades, several works have been published in the area of Byzantine FaultTolerance (BFT) [61, 70, 133]. The problem of computing (correctly) in spite of failures isin fact challenging and important for the integrity of systems, for example web services anddatabases, that are constantly under attack [128]. Hence, much effort has been employed to findefficient solutions to several coordination problems (such as reliable broadcast, consensus, etc.)in different scenarios, particularly in relation to BFT-SMR [73, 110].

A common feature that characterizes these SMR-based works is the adopted failure model.It specifies that out of a number n of machines only a constant fraction of them experiencearbitrary failures. Typical numbers are: n = 5f + 1 [19, 82] with fast communication primitivesthat need many replicas; n = 3f+1 (resilient-optimal) [24, 70, 131] with slower yet more robustcommunication primitives; n = 2f + 1 [6, 133] assuming the existence of a trusted module. Inall cases, the paradigm enables correct distributed computing algorithms in spite of f failures.

In the context of PR, Remus [40, 90] leverages hardware virtualization to build highly avail-able systems. The entire state of a virtual machine is replicated asynchronously to backup repli-cas and external output is not released until the propagation completes. Remus however onlytolerates fail-stop failures.

Today’s platforms are way more capable than simple processing units or specialized devices,such as clocks [134]; also, trusted components do not just store cryptographic keys and digitallysign messages [6, 33, 38, 75, 133]. We build on these works to guarantee the properties of trans-

26

ferable authentication and non-equivocation [36], which allow to reduce the overall number ofreplicas in the system. However, we further leverage the capability of current trusted componentsto provide a proof of work that can be trusted. This allows us to reduce the number of replicatedexecutions and to increase efficiency.

TrustedDB [16] bears similarities with our work in that it uses an IBM secure coprocessorto execute the privacy-sensitive part of a database. However, our objective is to secure gen-eral applications and to increase the efficiency of fault-tolerant replication by means of efficientexecution verification. Additionally, our secure protocols are based on an abstract trusted com-ponent, whose implementation can possibly be based on a TPM (i.e., about 200× cheaper than asecure coprocessor).

4.3 Finding a Niche to Fit Trusted Computing

We show how the security and efficiency gap liying between AR and PR can be bridged by usinghardware-based security techniques.Failure Masking vs. Resource-Efficiency A distributed service S may experience several typesof failures that can undermine its correctness. Among these, Byzantine [73] failures representthe most general class, as no assumptions are made about the correctness of S’s execution on aplatform. In fact, its observable actions (i.e., messages) — which represent the only informationthat affects the behavior of other participants in the system — are allowed to be arbitrary, suchas: none, as if the platform crashed; corrupted, for instance due to a bit flip in the network; ormalicious, for instance due to a software attack.

The first two types can be easily addressed. Crash failures can be masked by a variety ofpassive and active replication techniques, while message corruptions can be detected by usingerror detection codes or checksums.

Malicious failures that compromise S’s execution are instead harder to detect. If only a singleinstance of S is executed (e.g., PR), it is difficult to distinguish whether the messages it generatesbelong to a correct S instance or to some malicious code that impersonates S.

Solutions to deal with such failures involve masking or checking them through redundantexecutions. BFT-SMR addresses this problem by fully replicating the service [12, 30, 35, 38, 67,70, 133, 135]. Since multiple replicas run S and a majority of them are assumed correct, then thecorrect response can be voted: the majority of messages with a matching response wins. Othersolutions [59, 88] use trusted auditors (i.e., users that raise requests and then run S themselvesto compare that the received results match). However, besides requiring S’s re-execution, theyalso assume that the auditors are not recognizable, which makes the technique probabilistic (i.e.,not all replies are checked), inefficient (due to the additional required resources) and ineffectiveif the auditors get spotted.

This highlights a fundamental tradeoff between failure masking and resource efficiency.While PR can protect against certain failures, it cannot handle malicious failures since thereis only a single execution of the service. In contrast, to deal with arbitrary failures, AR has toexecute an instance of the service on each replica, making it inherently inefficient. In addition, it

27

needs to run a more complex protocol, such as consensus [30], to coordinate the replicas.Failure Detection: a Niche for Trusted Computing. The previous tradeoff is due to thedifficulty of checking a service execution. Hardware-based security works just in this directionto make execution verification: simple, by adding support for secure code execution; efficient,by avoiding re-executions; and trustworthy, by reducing the trusted computing base and makingprecise trust assumptions. Execution verification therefore enables the detection of an incorrectservice execution. We leverage such capability in our novel replicated system.

4.4 Completed Work

We introduce the concept of Verified Passive Replication (V-PR) that brings the AR and PRworlds together through the computation verification technique. Our V-PR scheme attains secu-rity guarantees as in BFT-SMR and resource-efficiency as in PR, without trading off generality.In particular, V-PR’s secure operations exploit novel features of recent, commercially availableand widely spread trusted components, whose capabilities are expected to be further enhancedand extended in future processor architectures (e.g., Intel SGX [64]). Resource-efficiency resultsfrom the reduced demand of computational resources of PR systems. V-PR is fully passive asit replaces the replicated execution of a service with a cheap replicated procedure that verifiesthe output of a single execution flow at the primary replica. Finally, generality is achieved intwo forms. First, V-PR does not make any assumptions on the service running above it. Sec-ond, it does not impose the implementation of a deterministic service, since non-deterministicoperations are allowed by construction.

This approach greatly departs from previous works. So far the main objective has been toenhance the efficiency by improving the coordination protocols [39, 58, 70, 82], by reducing thenumber of executing replicas [43, 66, 135, 138], or by reducing the overall number of replicas[6, 33, 38, 133]. Although improvements are significant, to the best of our knowledge, in allprevious works the number of parallel executions that are performed is still linear in the numberof available replicas. V-PR cuts down the execution effort to one, simplifies the coordinationprotocols and enables non-deterministic executions.

4.4.1 Overview

The general structure of V-PR is based on a typical passively replicated system [7, 25, 40, 42](see Fig. 4.1). A primary replica accepts authenticated requests from clients and it is responsi-ble for the trusted execution of the requested service. If the request is read-only—it does notupdate the system’s state—then a response is immediately returned to the client. Otherwise, theprimary first provides the backup replicas with authenticated state updates that are necessary toguarantee durability and waits for them to complete. The backup replicas do not execute theservice but verify and apply the updates to their current state, so to reach the same state as theprimary. Eventually, they return an authenticated acknowledgement. The primary checks theacknowledgments and releases the authenticated reply to the client.

28

Service'

AM' SM'RM'Manager'

10. reply

OS'

Security'MW'

Service'Client'

2. request

1. request

9. reply

U6Manager'

5. request 6.reply

'''''''TCC'' OS'3. request 8. reply

4. request 7. reply

'' ''TCC''

client' primary'replica' backup'replica'

U6Manager'Oth.Svcs'

AM' SM' RM'Manager'

OS'7a. update 7e. ack

7b. update 7d. ack

7c. trusted update

Network'

Oth.Svcs'

Update'Service'

Oth.Svcs'

Level'Applica6on'

Middleware'

SW/HW''support'

Update'

Figure 4.1: Architecture of the Verified Passive Replication scheme. Light shaded parts correspond to the trustedsystem components.

In V-PR, the service execution flow is protected by means of a trusted computing compo-nent (henceforth TCC) available in each replica. The service is run on top of the TCC (notthe OS), which isolates the service execution from any external (possibly malicious) interfer-ence, including the OS. The TCC implements memory isolation mechanisms, for instance usinghardware-backed virtualization technology, to deny access to any software that attempts to by-pass its primitives. The TCC also vouches for and attests to the correctness of the execution, sothe client knows the correct service code was executed on the right data.

This execution architecture presents three main benefits from a security perspective. First,it minimizes the Trusted Computing Base (TCB) that comprises the hardware and the softwarethat support the service execution. In particular, it excludes the OS and other services it mayrun. Second, it provides a meaningful identification mechanism (using a hash) of the code runin the TCB. Third, it builds a chain of trust, based on strong cryptography, that directly links theoutputs of the service to its execution, and ultimately to the trusted hardware module.

Figure 4.1 shows the software running on the client, the primary replica, and the backupreplicas. The software includes the OS, the TCC, the V-PR software, the client, the serviceon the primary and available on the backups, and other running services. Both the client andservice are unmodified general applications. The software running in the trusted environmentis shaded. The core components of V-PR are: the client’s Security MiddleWare (SMW), thetrusted Manager(s), the untrusted U-Manager(s), and the State Update Service. The SMW isa basic security layer that sets up a secure communication channel between the client and theprimary’s trusted environment. The Manager is implemented on top of the TCC and it providesa minimal interface to execute the service and to set up communication channels with the clientsand the other replicas. The Manager is divided in 3 parts: the Application Manager (AM) dealsdirectly with the service; the Security Manager (SM) is responsible for key management andauthentication; and the Replication Manager (RM) implements part of the replication logic. Theuntrusted U-Manager is a simple application implemented on top of the OS that mainly handlesnetwork communications, i.e., communication between Managers, the SMW, and Update Serviceflows through the U-Manager. The Update Service is a tiny module that is used by the backupreplicas to apply the updates received from the primary to the current state.

29

4.4.2 Model

The system consists of a group of n nodes, each one equipped with a TCC. The V-PR Managerand the replicated service run in the protected environment provided by the TCC. The rest, in-cluding the V-PR U-Manager, the OS and other services, execute in the untrusted part of the node.An arbitrary number of clients can access the system, but they need to be enrolled beforehandto obtain the necessary authentication credentials (e.g., by contacting an identity managementservice).

The TCC only runs code on-demand, when it is explicitly called by an untrusted softwarecomponent. The component can give some data as input and in the end receives the output value.While executing, the code is isolated with no access to the network or general storage (e.g., thedisk). It can however use a limited set of secure primitives to create counters, encrypt data,or perform attestation. Moreover, any keys that are generated by the code cannot be observed,unless they are returned unprotected to the calling component.

At most a minority f = bn−12c of the nodes can fail. In particular, the TCC can only suffer

crashes but the rest of the system and network can experience Byzantine failures. Therefore, thecode in the TCC either produces correct results or no values. Untrusted components (such as theU-Manager) can corrupt data, delay the execution or do any other attempt to maliciously breakthe protocol. Messages may be modified, removed or delayed. These assumptions are similar toother systems based on trusted components [33, 38, 66, 75, 133].

The V-PR protocol ensures safety in the asynchronous model. Liveness is guaranteed in peri-ods of partial synchrony—when messages are delivered and processed within a fixed but possiblyunknown time bound. This can be achieved through retransmissions and acknowledgments.

4.4.3 Evaluation

Implementation. We implement a message passing subsystem for V-PR based on ZeroMQ[139], a high-performance messaging library that abstracts details of the underlying socket im-plementation. Also, we implement a TCC based on XMHF-TrustVisor [85, 130].Comparison. We compared the V-PR middleware against two open-source BFT-SMR libraries,namely BFT-SMaRt [116] and Prime [10] (Tab. 4.1). The exchanged messages measure thecoordination overhead to maintain state consistency (as shown in [30]). Each message exchangephase counts as one hop. V-PR outperforms BFT-SMaRt and Prime because it avoids expensivecoordination protocols for request ordering, such as Consensus.

BFT-SMaRt Prime V-PR(r)

Messages (w)1+1

4+3+16+16+4 3+16+16+3+12+161+1

1+3+3+1Hops (r) 2 (w) 5 6 (r) 2 (w) 4Replicas (f=1) 4 4 3

Executions4

(active)4

(active)1 (active)

2 (passive)Table 4.1: Normal request execution in BFT-SMaRt, Prime and V-PR.

30

Experimental analysis using a simple zero-overhead service in fault-free runs however showsthat V-PR is slower than BFT-SMaRt for single requests. This is primarily due to the TCC. Primeis instead mainly delayed by the heavy use of signatures. However, all systems take advantage ofrequest batching to improve their efficiency. Noticeably, V-PR matches BFT-SMaRt latency—which is already highly optimized—when the batch size reaches around 12 requests.Computational efficiency. We performed application-level measurements to understand thecomputational savings of V-PR. The results are encouraging since the V-PR system only per-forms one service execution and two deterministic backup services, while BFT-SMR runs fourredundant service copies. In our workload of write requests, backup operations are 5-6× fasterthan the service execution. This allows to save about 3× the CPU resources (overall, in thedistributed system) required for the robustly replicated service.Main overhead and future TCCs. The main overhead is due to the implementation of theTCC, which is not optimized. The implementation incurs several context switches due to virtu-alization operations, code identification and data transfers between the untrusted and the isolatedenvironment.

Future trusted computing technology is expected to reduce TCC costs at the hardware level,and can be easily integrated below V-PR. In fact, V-PR is not bound to XMHF-TrustVisor, sinceit is built on an abstract TCC that hides the technical detail of the underlying trusted component.Another TCC could be based for instance on Intel SGX [64], which provides on-CPU trustedcomputing services based on CPU instructions rather than hypercalls.

4.5 Impact of Large Code Execution and Large State

We briefly discuss how the work in previous sections (Sections 2 and 3) impacts our replicationscheme.Efficient code execution. Our current implementation is based on the measure-once-execute-once paradigm (Section 2.2) which is not efficient for large-sized services. This contributes tothe slowdown, with respect to other replication schemes, that is noticeable for single requests.Other replication schemes in fact do not suffer such overhead and, besides, they are optimized[116].

Our contribution in Section 2 is important to provide fresh cost-effective integrity guaranteesso to increase the trustworthiness of the primary’s execution. Our V-PR scheme is in fact heavilydependent on the secure execution of the primary service. Also, V-PR does not impose restric-tions on the service implementation, that could be larger and reacher than the service in our casestudy.Efficient handling of a large state. Our current implementation is based on the traditionalapproach of loading the whole input required for the computation in the isolated environment. Itis clear that this is secure, and also efficient and reasonable for small computations over a smallstate. Our case study in fact uses a small database of a few hundred items.

Such a design is however detrimental for more realistic applications. For example, a multi-gigabyte database would take long time to load, and the process would consume lots of resources

31

(both CPU and memory). Additionally, it might happen that only a small part of the state isaccessed, for example to serve the requests of a few clients.

Our contribution in Section 3 is thus important to improve efficiency without sacrificingsecurity. The state is loaded and authenticated as required by the service. The ultimate objectiveis to improve resource utilization and performance.

4.6 Proposed Work

We propose a formal treatment of the V-PR scheme to prove its correctness. Although V-PR’sobjective is to increase the security in terms of availability of secure outsourced services, as wellas the efficiency of replicated systems, unfortunately, it has only been analyzed informally. Webelieve this is feasible because extensive literature is available with proofs of correctness [13, 30,91], albeit for active replication, with which however passive replication bears commonalities[103]. Additionally, a formal proof would significantly strengthen our findings.

Second, as a stretch goal, it would be interesting to reason about the minimal assumptionsthat are required in our model to achieve efficient and robust replication. For example, fromV-PR it appears that the trusted environment only needs a small amount of stable storage to storethe secret keys of the trusted component. If so, the remaining memory can be volatile and canbe theoretically reset after each trusted execution. This would prove the possibility of achievingresilient executions in spite of failures using almost-memoryless processes.

Aside from the advances in availability of outsourced services under minimal assumptions,our proposed work also provides contributions to the Distributed Computing area. To the bestof our knowledge, the execution verification capability, which is the key that makes V-PR fea-sible, has not been investigated in previous works. Previous works [6, 38, 133] in fact onlyused a trusted component to ensure non-equivocation and transferable authentication [36] that is,roughly speaking, as a trusted anchor to sign messages.

32

Section 5

Timeline

Timeline

2015 2016Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug

Submit-§2

Large State-§3

Prototype-§3

New applications-§2

Modeling&Correctness-§4

Logic&Correctness-§2

Thesis writing

Defense

Milestones:• [early Dec’15] Submission of completed work (§2) to DSN/AsiaCCS 2016• [Jan 2016] Complete design and prototype in §3, and submit to Usenix ATC/Security 2016• [Jan’16-Feb’16] Investigate new applications of §2, namely in-network Middleboxes (pos-

sible target conference: HotSec’16)• [Feb’16-Apr’16] Model and correctness of V-PR (possible target conference: DISC’16)• [Apr’16-Jun’16] Sound logic to analyze belief propagation through explicitly trusted prin-

cipals §2 (possible target conference: SecureComm’16)• [Jul’16-Aug’16] Write thesis document• [Aug’16] Thesis defense

33

34

Bibliography

[1] SQLite in Android. http://developer.android.com/reference/ android/database/sqlite/SQLiteDatabase.html, . 2.4.3

[2] SQLite Deployments. http://sqlite.org/mostdeployed.html, . 2.4.3

[3] SQLite in iCloud. https://developer.apple.com/library/ios/ documentation/DataManagement/Conceptual/UsingCoreData Withi-CloudPG/UsingSQLiteStoragewithi Cloud/UsingSQLiteStoragewithiCloud.html, . 2.4.3

[4] Martın Abadi and Mark R. Tuttle. A semantics for a logic of authentication (extended abstract). In Proceedings of the 10th ACMsymposium on Principles of Distributed Computing (PODC), pages 201–216, 1991. 1.3, 2.5, 4

[5] Martın Abadi and Ted Wobber. A Logical Account of NGSCB. Formal Techniques for Networked and Distributed (FORTE), pages 1–12,2004. 1.3

[6] Ittai Abraham, Marcos Kawazoe Aguilera, and Dahlia Malkhi. Fast Asynchronous Consensus with Optimal Resilience. In Proceedingsof the 24th Conference on Distributed Computing (DISC), pages 4–19, 2010. 1.3, 4.2, 4.4, 4.6

[7] Peter A. Alsberg and John D. Day. A principle for resilient sharing of distributed resources. In In Proc. of the 2nd International Conferenceon Software Engineering (ICSE), page 562, 1976. 4.1, 4.4.1

[8] Amazon. Amazon EC2. http://aws.amazon.com/ec2/, . 1

[9] Amazon. AWS CloudHSM, Secure Key Storage and Cryptographic Operations. http://aws.amazon.com/cloudhsm, . 1.3

[10] Y Amir, B Coan, J Kirsch, and J Lane. Prime: Byzantine Replication under Attack. IEEE Transactions on Dependable and SecureComputing, 8(4):564–577, jul 2011. 4.4.3

[11] Joao Antunes and Nuno Neves. DiveInto: Supporting Diversity in Intrusion-Tolerant Systems. In Proceedings of the 30th Symposium onReliable Distributed Systems (SRDS), pages 137–146, 2011. 4.1

[12] Pierre-Louis Aublin, Sonia Ben Mokhtar, and Vivien Quema. RBFT: Redundant Byzantine Fault Tolerance. In Proceedings of the 33rdInternational Conference on Distributed Computing Systems (ICDCS), pages 297–306, 2013. 4.3

[13] Pierre-Louis Aublin, Rachid Guerraoui, Nikola Knezevic, Vivien Quema, and Marko Vukolic. The Next 700 BFT Protocols. ACMTransactions on Computer Systems (TOCS), 32(4):1–45, jan 2015. 4.6

[14] Ahmed M. Azab, Peng Ning, and Xiaolan Zhang. SICE: a hardware-level strongly isolated computing environment for x86 multi-coreplatforms. In Proceedings of the 18th ACM conference on Computer and communications security (CCS), page 375, 2011. ISBN9781450309486. doi: 10.1145/2046707.2046752. 1.3

[15] Sumeet Bajaj and Radu Sion. TrustedDB. In Proceedings of the international Conference on Management of Data (SIGMOD), page 205,2011. ISBN 9781450306614. doi: 10.1145/1989323.1989346. 1.3, 2.2, 3.1, 3.3

[16] Sumeet Bajaj and Radu Sion. TrustedDB: A Trusted Hardware-Based Database with Privacy and Data Confidentiality. IEEE Transactionson Knowledge and Data Engineering, 26(3):752–765, mar 2014. 2.2, 4.2

[17] Andrew Baumann, Marcus Peinado, and Galen Hunt. Shielding applications from an untrusted cloud with Haven. In Proceedings of the11th USENIX conference on Operating Systems Design and Implementation (OSDI), pages 267–283, oct 2014. ISBN 978-1-931971-16-4.1.3, 2.2, 2.2, 2.3

[18] Mihir Bellare, Oded Goldreich, and Shafi Goldwasser. Incremental cryptography and application to virus protection. In Proceedings ofthe 27th ACM Symposium on Theory of Computing (STOC), pages 45–56, New York, New York, USA, may 1995. ACM Press. 3.2

[19] Michael Ben-Or. Another advantage of free choice: Completely asynchronous agreement protocols. In Proceedings of the 2nd Symposiumon Principles of Distributed Computing (PODC), pages 27–30, 1983. 4.2

[20] John Black and Phillip Rogaway. A Block-Cipher Mode of Operation for Parallelizable Message Authentication. pages 384–397, may2002. 3.2

[21] Manuel Blum, W. Evans, P. Gemmell, Sampath Kannan, and Moni Naor. Checking the correctness of memories. In Proceedings 32ndAnnual Symposium of Foundations of Computer Science (FOCS), pages 90–99, sep 1991. 3.2

[22] Sara Bouchenak, Gregory Chockler, Hana Chockler, Gabriela Gheorghe, Nuno Santos, and Alexander Shraer. Verifying cloud services.ACM SIGOPS Operating Systems Review, 47(2):6, jul 2013. 1.3

35

[23] Colin Boyd and Wenbo Mao. On a limitation of BAN logic. In Proceeding of the Workshop on the theory and application of cryptographictechniques on Advances in cryptology (Eurocrypt), pages 240–247. Springer-Verlag New York, Inc., jan 1994. 2.5

[24] Gabriel Bracha. An asynchronous [(n - 1)/3]-resilient consensus protocol. In Proceedings of the 3rd Symposium on Principles ofDistributed Computing (PODC), pages 154–162, 1984. 4.2

[25] Navin Budhiraja, Keith Marzullo, F. B. Schneider, and Sam Toueg. The primary-backup approach. In Distributed systems (2nd Ed.),pages 199–216. may 1993. 4.1, 4.4.1

[26] Sven Bugiel and Jan-Erik Ekberg. Implementing an application-specific credential platform using late-launched mobile trusted module.In Proceedings of the 5th ACM workshop on Scalable trusted computing (STC), page 21, 2010. ISBN 9781450300957. doi: 10.1145/1867635.1867641. 2.3

[27] Michael Burrows, Martın Abadi, and Roger Needham. A Logic of Authentication. Technical report, Digital Equipment Corp., 1989. 1.3,2.5

[28] Michael Burrows, Martın Abadi, and Roger Needham. A Logic of Authentication. ACM Transactions on Computer Systems (TOCS), 8(1):18–36, feb 1990. ISSN 07342071. doi: 10.1145/77648.77649. 1.3, 2.5

[29] Jan Camenisch and Anna Lysyanskaya. Dynamic Accumulators and Application to Efficient Revocation of Anonymous Credentials. InProceedings of the 22nd International Conference on Advances in Cryptology (CRYPTO), pages 61–76. Springer-Verlag, aug 2002. 3.2

[30] Miguel Castro and Barbara Liskov. Practical byzantine fault tolerance and proactive recovery. ACM Transactions on Computer Systems(TOCS), 20(4):398–461, nov 2002. ISSN 07342071. doi: 10.1145/571637.571640. URL http://portal.acm.org/citation.cfm?doid=571637.571640. 4.1, 4.3, 4.4.3, 4.6

[31] Stephen Checkoway and Hovav Shacham. Iago attacks. In Proceedings of the 18th international conference on Architectural support forprogramming languages and operating systems (ASPLOS), volume 41, page 253, mar 2013. 1.3

[32] Yuqun Chen, Ramarathnam Venkatesan, Matthew Cary, Ruoming Pang, Saurabh Sinha, and Mariusz H. Jakubowski. Oblivious Hashing:A Stealthy Software Integrity Verification Primitive. In Proceedings of the 5th International Workshop on Information Hiding (IH), pages400–414, oct 2002. ISBN 3-540-00421-1. 2.3

[33] Byung-Gon Chun, Petros Maniatis, Scott Shenker, and John Kubiatowicz. Attested append-only memory: making adversaries stick totheir word. In Proceedings of 21st Symposium on Operating Systems Principles (SOSP), volume 41, page 189, 2007. URL http://portal.acm.org/citation.cfm?id=1323293.1294280. 1.3, 4.2, 4.4, 4.4.2

[34] Dwaine Clarke, G. Edward Suh, Blaise Gassend, Ajay Sudan, Marten van Dijk, and Srinivas Devadas. Towards Constant BandwidthOverhead Integrity Checking of Untrusted Data. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), pages 139–153.IEEE Computer Society, may 2005. ISBN 0-7695-2339-0. 3.2

[35] Allen Clement, Edmund L. Wong, Lorenzo Alvisi, Mike Dahlin, and Mirco Marchetti. Making Byzantine fault tolerant systems tolerateByzantine faults. Proceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation (OSDI), pages 153–168,2009. URL http://portal.acm.org/citation.cfm?id=1558988. 4.1, 4.3

[36] Allen Clement, Flavio Junqueira, Aniket Kate, and Rodrigo Rodrigues. On the (limited) power of non-equivocation. In Proceedingsof the Symposium on Principles of Distributed Computing (PODC), page 301, jul 2012. URL http://dl.acm.org/citation.cfm?id=2332432.2332490. 4.2, 4.6

[37] Vinicius Cogo, Alysson Bessani, Francisco Couto, and Paulo Verıssimo. A High-Throughput Method to Detect Privacy-Sensitive HumanGenomic Data. In Prodeedings of the Workshop on Privacy in the Electronic Society (WPES), 2015. 3.1, 3.3

[38] Miguel Correia, Nuno Neves, and Paulo Verissimo. How to Tolerate Half Less One Byzantine Nodes in Practical Distributed Systems.In Proceedings of the 23rd IEEE International Symposium on Reliable Distributed Systems (SRDS), pages 174–183, 2004. 1.3, 4.1, 4.2,4.3, 4.4, 4.4.2, 4.6

[39] James Cowling, Daniel Myers, Barbara Liskov, Rodrigo Rodrigues, and Liuba Shrira. HQ replication: a hybrid quorum protocol forbyzantine fault tolerance. In Proceedings of the 7th Symposium on Operating Systems Design and Implementation (OSDI), pages 177–190, 2006. ISBN 1-931971-47-1. 4.4

[40] Brendan Cully, Geoffrey Lefebvre, Dutch Meyer, Mike Feeley, Norm Hutchinson, and Andrew Warfield. Remus: high availability viaasynchronous virtual machine replication. In Proceedings of the 5th Symposium on Networked Systems Design and Implementation(NSDI), pages 161–174, apr 2008. ISBN 111-999-5555-22-1. 4.2, 4.4.1

[41] Anupam Datta, Jason Franklin, D. Garg, and Dilsun K Kaynar. A logic of secure systems and its application to trusted computing. InProceedings of the 30th IEEE Symposium on Security and Privacy (S&P), pages 221–236, 2009. 1.3, 2.5

[42] Xavier Defago, A. Schiper, and N. Sergent. Semi-passive replication. In Proceedings of the 17th IEEE Symposium on Reliable DistributedSystems (SRDS), pages 43–50, 1998. ISBN 0-8186-9218-9. doi: 10.1109/RELDIS.1998.740473. 4.4.1

[43] T Distler, I Popov, and W Schroder-Preikschat. SPARE: Replicas on Hold. In Proceedings of the Network & Distributed System SecuritySymposium (NDSS), 2011. 4.4

[44] Alan M. Dunn, Owen S. Hofmann, Brent Waters, and Emmett Witchel. Cloaking malware with the trusted platform module. In Proceed-ings of the 20th USENIX Conference on Security, page 26. USENIX Association, aug 2011. URL http://dl.acm.org/citation.cfm?id=2028067.2028093. 1.3

[45] Cynthia Dwork, Moni Naor, Guy N. Rothblum, and Vinod Vaikuntanathan. How Efficient Can Memory Checking Be? In Omer Reingold,editor, Proceedings of the 6th Theory of Cryptography Conference (TCC), volume 5444 of Lecture Notes in Computer Science, pages

36

503–520, Berlin, Heidelberg, feb 2009. Springer Berlin Heidelberg. ISBN 978-3-642-00456-8. doi: 10.1007/978-3-642-00457-5. URLhttp://dl.acm.org/citation.cfm?id=1530441.1530471. 3.2

[46] Jan-Erik Ekberg, N. Asokan, Kari Kostiainen, and Aarne Rantala. Scheduling execution of credentials in constrained secure environments.In Proceedings of the 3rd ACM Workshop on Scalable Trusted Computing (STC), page 61, 2008. ISBN 9781605582955. doi: 10.1145/1456455.1456465. 2.3

[47] Paul England and Marcus Peinado. Authenticated Operation of Open Computing Devices. In Proceedings of the 7th Australian Confer-ence on Information Security and Privacy (ACISP), pages 346–361, jul 2002. 1.3, 2.3

[48] Paul England, Butler Lampson, J. Manferdelli, Marcus Peinado, and B. Willman. A Trusted Open Platform. Computer, 36(7):55–62,2003. ISSN 0018-9162. doi: 10.1109/MC.2003.1212691. 1.3, 2.3

[49] Marc Fischlin. Incremental cryptography and memory checkers. In Proceedings of the 16th annual international conference on Theoryand application of cryptographic techniques (EUROCRYPT), pages 393–408. Springer-Verlag, may 1997. ISBN 3-540-62975-0. URLhttp://dl.acm.org/citation.cfm?id=1754542.1754579. 3.2

[50] Rui Garcia, Rodrigo Rodrigues, and Nuno Preguica. Efficient middleware for byzantine fault tolerant database replication. In Proceedingsof the 6th European conference on Computer systems (EuroSys), pages 107–122, 2011. ISBN 9781450306348. doi: 10.1145/1966445.1966456. 4.1

[51] Blaise Gassend, G. Edward Suh, Dwaine Clarke, Marten van Dijk, and Srinivas Devadas. Caches and Hash Trees for Efficient MemoryIntegrity Verification. In Proceedings of the 9th International Symposium on High-Performance Computer Architecture (HPCA), page 295.IEEE Computer Society, feb 2003. ISBN 0-7695-1871-0. URL http://dl.acm.org/citation.cfm?id=822080.822806.3.2

[52] Rosario Gennaro, Craig Gentry, and Bryan Parno. Non-interactive verifiable computing: outsourcing computation to untrusted workers.In Proceedings of the 30th Annual Conference on Advances in Cryptology (CRYPTO), pages 465–482, aug 2010. ISBN 3-642-14622-8,978-3-642-14622-0. 1

[53] Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung. The Google file system. In Proceedings of the 19th Symposium on OperatingSystems Principles (SOSP), volume 37, page 29. ACM, dec 2003. ISBN 1-58113-757-5. doi: 10.1145/1165389.945450. URL http://dl.acm.org/citation.cfm?id=1165389.945450. 4.1

[54] Michael T. Goodrich, Roberto Tamassia, and Jasminka Hasic. An Efficient Dynamic and Distributed Cryptographic Accumulator. InProceedings of the 5th International Conference on Information Security (ISC), pages 372–388. Springer-Verlag, sep 2002. ISBN 3-540-44270-7. URL http://dl.acm.org/citation.cfm?id=648026.744658. 3.2

[55] David Grawrock. The Intel Safer Computing Initiative: Building Blocks for Trusted Computing. Engineer to Engineer Series. Intel Press,2006. ISBN 9780976483267. URL http://books.google.com/books?id=WmGjSgAACAAJ. 1.3

[56] David Grawrock. Dynamics of a Trusted Platform: A Building Block Approach. Intel Press, apr 2009. ISBN 1934053171,9781934053171. 1.3

[57] Dov Greenbaum, Andrea Sboner, Xinmeng Jasmine Mu, and Mark Gerstein. Genomics and privacy: implications of the new reality ofclosed data for the field. PLoS computational biology, 7(12):e1002278, dec 2011. ISSN 1553-7358. doi: 10.1371/journal.pcbi.1002278.URL http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002278. 3.1

[58] Rachid Guerraoui, Nikola Knezevic, Vivien Quema, and Marko Vukolic. The next 700 BFT protocols. In Proceedings of the 5th Europeanconference on Computer systems (EuroSys), page 363, 2010. ISBN 9781605585772. doi: 10.1145/1755913.1755950. 4.1, 4.4

[59] Andreas Haeberlen, Petr Kouznetsov, and Peter Druschel. PeerReview: Practical accountability for distributed systems. In Proceedingsof the Symposium on Operating Systems Principles (SOSP), pages 175–188, 2007. 4.3

[60] WE Hall and CS Jutla. Parallelizable authentication trees. Selected Areas in Cryptography, 2006. URL http://link.springer.com/chapter/10.1007/11693383_7. 3.2

[61] James Hendricks, Shafeeq Sinnamohideen, Gregory R. Ganger, and Michael K. Reiter. Zzyzx: Scalable fault tolerance through Byzantinelocking. In IEEE/IFIP International Conference on Dependable Systems & Networks (DSN), pages 363–372, jun 2010. ISBN 978-1-4244-7500-1. doi: 10.1109/DSN.2010.5544297. URL http://ieeexplore.ieee.org/lpdocs/epic03/wrapper.htm?arnumber=5544297. 4.2

[62] Patrick Hunt, Mahadev Konar, Flavio P. Junqueira, and Benjamin Reed. ZooKeeper: wait-free coordination for internet-scale systems. InProceedings of the USENIX Annual Technical Conference (ATC), page 11, jun 2010. URL http://dl.acm.org/citation.cfm?id=1855840.1855851. 4.1

[63] IBM. IBM PCI-e Cryptographic Coprocessor. http://www-03.ibm.com/security/cryptocards/pciecc. 1.3

[64] Intel. Software Guard Extensions. https://software.intel.com/sites/ default/files/managed/48/88/329298-002.pdf, . 1.2, 1.3, 2.2, 2.2, 2.3,2.4.1, 4.4, 4.4.3

[65] Intel. Intel Trusted Execution Technology. http://www.intel.com/ content/dam/www/ public/us/en/documents/ guides/intel-txt -software-development-guide.pdf, . 1.3, 1, 2.3, 3.1

[66] Rudiger Kapitza, Johannes Behl, Christian Cachin, Tobias Distler, Simon Kuhnle, Seyed Vahid Mohammadi, Wolfgang Schroder-Preikschat, and Klaus Stengel. CheapBFT: resource-efficient byzantine fault tolerance. In Proceedings of the 7th European Con-ference on Computer Systems (EuroSys), page 295, 2012. ISBN 9781450312233. doi: 10.1145/2168836.2168866. URL http://dl.acm.org/citation.cfm?id=2168836.2168866. 4.1, 4.4, 4.4.2

37

[67] Manos Kapritsos, Yang Wang, Vivien Quema, Allen Clement, Lorenzo Alvisi, and Mike Dahlin. All about Eve: execute-verify replicationfor multi-core servers. In Proceedings of the 10th USENIX conference on Operating Systems Design and Implementation (OSDI), pages237–250, oct 2012. 4.1, 4.3

[68] Patrick Koeberl, Steffen Schulz, Ahmad-Reza Sadeghi, and Vijay Varadharajan. TrustLite. In Proceedings of the 9th European Conferenceon Computer Systems (EuroSys), pages 1–14, 2014. ISBN 9781450327046. doi: 10.1145/2592798.2592824. 1.3, 2.3

[69] Kari Kostiainen, Jan-Erik Ekberg, N. Asokan, and Aarne Rantala. On-board credentials with open provisioning. In Proceedings of the 4thInternational Symposium on Information, Computer, and Communications Security (ASIACCS), page 104, 2009. ISBN 9781605583945.doi: 10.1145/1533057.1533074. 2.3

[70] Ramakrishna Kotla, Lorenzo Alvisi, Mike Dahlin, Allen Clement, and Edmund L. Wong. Zyzzyva: Speculative Byzantine Fault Toler-ance. In Proceedings of the 21st Symposium on Operating Systems Principles (SOSP), volume 41, pages 45–58, 2007. 4.2, 4.3, 4.4

[71] Leslie Lamport. Time, clocks, and the ordering of events in a distributed system. Communications of the ACM, 21(7):558–565, 1978. 1,4.1

[72] Leslie Lamport. The part-time parliament. ACM Transactions on Computer Systems (TOCS), 16(2):133–169, may 1998. ISSN 07342071.doi: 10.1145/279227.279229. URL http://portal.acm.org/citation.cfm?id=279227.279229. 4.1

[73] Leslie Lamport, Robert Shostak, and Marshall Pease. The Byzantine Generals Problem. ACM Transactions on Programming Languagesand Systems (TOPLAS), 4(3):382–401, 1982. ISSN 0164-0925. URL http://portal.acm.org/citation.cfm?id=357176.1, 4.1, 4.2, 4.3

[74] Butler Lampson, Martın Abadi, Michael Burrows, and Edward Wobber. Authentication in distributed systems: theory and practice. ACMTransactions on Computer Systems (TOCS), 10(4):265–310, nov 1992. ISSN 07342071. doi: 10.1145/138873.138874. 2.3

[75] Dave Levin, John R. Douceur, Jacob R. Lorch, and Thomas Moscibroda. TrInc: small trusted hardware for large distributed systems. InProceedings of the 6th USENIX Symposium on Networked Systems Design and Implementation (NSDI), pages 1–14, 2009. 4.2, 4.4.2

[76] Heng Li, Bob Handsaker, Alec Wysoker, Tim Fennell, Jue Ruan, Nils Homer, Gabor Marth, Goncalo Abecasis, and Richard Durbin. TheSequence Alignment/Map format and SAMtools. Bioinformatics (Oxford, England), 25(16):2078–9, aug 2009. 3.3

[77] Jiangtao Li, Ninghui Li, and Rui Xue. Universal Accumulators with Efficient Nonmembership Proofs. In Jonathan Katz and Moti Yung,editors, Proceedings of the 5th international conference on Applied Cryptography and Network Security (ACNS), volume 4521 of LectureNotes in Computer Science, pages 253–269, Berlin, Heidelberg, jun 2007. Springer Berlin Heidelberg. ISBN 978-3-540-72737-8. doi:10.1007/978-3-540-72738-5. URL http://dl.acm.org/citation.cfm?id=1419765.1419788. 3.2

[78] Hans Lohr, HariGovind V. Ramasamy, Ahmad-Reza Sadeghi, Stefan Schulz, Matthias Schunter, and Christian Stuble. Enhancing gridsecurity using trusted virtualization. In Proceedings of the 4th international conference on Autonomic and Trusted Computing (ATC),pages 372–384. Springer-Verlag, jul 2007. ISBN 3-540-73546-1, 978-3-540-73546-5. URL http://dl.acm.org/citation.cfm?id=2394798.2394846. 1.3

[79] John MacCormick, Nick Murphy, Marc Najork, Chandramohan A. Thekkath, and Lidong Zhou. Boxwood: abstractions as the foundationfor storage infrastructure. In Proceedings of the 6th Symposium on Opearting Systems Design & Implementation (OSDI), page 8, dec2004. 4.1

[80] John Maccormick, Chandramohan A. Thekkath, Marcus Jager, Kristof Roomp, Lidong Zhou, and Ryan Peterson. Niobe: a PracticalReplication Protocol. Journal ACM Transactions on Storage (TOS), 3(4):1–43, feb 2008. 4.1

[81] Wenbo Mao, Andrew Martin, Hai Jin, and Huanguo Zhang. Innovations for Grid Security from Trusted Computing. In Bruce Christianson,Bruno Crispo, James A. Malcolm, and Michael Roe, editors, Security Protocols, volume 5087 of Lecture Notes in Computer Science,pages 132–149. Springer Berlin Heidelberg, Berlin, Heidelberg, oct 2009. ISBN 978-3-642-04903-3. doi: 10.1007/978-3-642-04904-0.URL http://dl.acm.org/citation.cfm?id=1692272.1692290. 1.3

[82] Jean-Philippe Martin and L. Alvisi. Fast Byzantine Consensus. In Proceedings of the International Conference on Dependable Systemsand Networks (DSN), pages 402–411, jun 2005. ISBN 0-7695-2282-3. doi: 10.1109/DSN.2005.48. URL http://www.computer.org/csdl/proceedings/dsn/2005/2282/00/22820402-abs.html. 4.2, 4.4

[83] Jonathan M. McCune, Bryan Parno, Adrian Perrig, Michael K. Reiter, and Hiroshi Isozaki. Flicker: An Execution Infrastructure for TCBMinimization. In Proceedings of the European Conference in Computer Systems (EuroSys), volume 42, pages 315–328, 2008. ISBN9781605580135. doi: 10.1145/1352592.1352625. 1.2, 2.2, 2.3, 2

[84] Jonathan M. McCune, Bryan Parno, Adrian Perrig, Michael K. Reiter, and Arvind Seshadri. How low can you go? ACM SIGPLANNotices, 43(3):14, mar 2008. ISSN 03621340. doi: 10.1145/1353536.1346285. 1.3

[85] Jonathan M. McCune, Yanlin Li, Ning Qu, Zongwei Zhou, Anupam Datta, Virgil Gligor, and Adrian Perrig. TrustVisor: Efficient TCBReduction and Attestation. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), pages 143–158, 2010. 1.2, 1.3, 2.2,1, 2.3, 2, 3.3, 3.1, 4.4.3

[86] Frank McKeen, Ilya Alexandrovich, Alex Berenzon, Carlos V. Rozas, Hisham Shafi, Vedvyas Shanbhogue, and Uday R. Savagaonkar.Innovative instructions and software model for isolated execution. In Proceedings of the 2nd Workshop on Hardware and ArchitecturalSupport for Security and Privacy (HASP), pages 1–1, 2013. ISBN 9781450321181. 1.3, 2.2

[87] Ralph C. Merkle. A Certified Digital Signature. In Proceedings on Advances in Cryptology (CRYPTO), pages 218–238, 1989. 3.2

[88] Nikolaos Michalakis, Robert Soule, and Robert Grimm. Ensuring content integrity for untrusted peer-to-peer content distribution net-works. In Proceeding of the 4th USENIX Conference on Networked Systems Design & Implementation (NSDI), page 11, apr 2007.

38

4.3

[89] Microsoft. BitLocker. http://windows.microsoft.com/en-us/windows7/products/features/bitlocker. 1.3

[90] Umar Farooq Minhas, Shriram Rajagopalan, Brendan Cully, Ashraf Aboulnaga, Kenneth Salem, and Andrew Warfield. RemusDB:transparent high availability for database systems. International Journal on Very Large Data Bases (VLDB Journal), 22(1):29–45, oct2013. 4.2

[91] Iulian Moraru, David G. Andersen, and Michael Kaminsky. There is more consensus in Egalitarian parliaments. In Proceedings of the24th ACM Symposium on Operating Systems Principles (SOSP), pages 358–372, nov 2013. 4.6

[92] Moni Naor and Guy N. Rothblum. The complexity of online memory checking. Journal of the ACM, 56(1):1–46, jan 2009. 3.2

[93] Brian M. Oki and Barbara H. Liskov. Viewstamped replication: a general primary copy. In Proceedings of the seventh annual ACMSymposium on Principles of distributed computing - PODC ’88, pages 8–17. ACM Press, jan 1988. ISBN 0897912772. doi: 10.1145/62546.62549. URL http://dl.acm.org/citation.cfm?id=62546.62549. 4.1

[94] Alina Oprea and Michael K. Reiter. Integrity checking in cryptographic file systems with constant trusted storage. In Proceedings of 16thUSENIX Security Symposium, page 13. USENIX Association, aug 2007. 3.2

[95] Emmanuel Owusu, Jorge Guajardo, Jonathan McCune, Jim Newsome, Adrian Perrig, and Amit Vasudevan. OASIS. In Proceedings ofthe 2013 Conference on Computer & Communications Security (CCS), pages 13–24, nov 2013. ISBN 9781450324779. doi: 10.1145/2508859.2516678. 1.3, 2.2, 2.3

[96] Bryan Parno and Craig Gentry. Pinocchio: Nearly practical verifiable computation. In Proceedings of the IEEE Symposium on Securityand Privacy (S&P), pages 238–252, 2013. 2.2

[97] Marshall Pease, R. Shostak, and Leslie Lamport. Reaching Agreement in the Presence of Faults. Journal of the ACM (JACM), 27(2):228–234, 1980. 4.2

[98] Raluca Ada Popa, Jacob R. Lorch, David Molnar, Helen J. Wang, and Li Zhuang. Enabling security in cloud storage SLAs with Cloud-Proof. In Proceedings of the USENIX Annual Technical Conference, page 31, jun 2011. 3.2

[99] Raluca Ada Popa, Catherine M. S. Redfield, Nickolai Zeldovich, and Hari Balakrishnan. CryptDB. In Proceedings of the 23rd ACMSymposium on Operating Systems Principles (SOSP), page 85, oct 2011. 3.1, 3.3

[100] Donald E. Porter, Silas Boyd-Wickizer, Jon Howell, Reuben Olinsky, and Galen C. Hunt. Rethinking the library OS from the top down. InProceedings of the 16th international conference on Architectural support for programming languages and operating systems (ASPLOS),volume 47, pages 291–304. ACM, jun 2012. 1.3

[101] Rackspace. Cloud Servers. http://www.rackspace.com/cloud/servers/. 1

[102] Jason F. Reid and William J. Caelli. DRM, trusted computing and operating system architecture. In roceedings of the Australasianworkshop on Grid computing and e-research (ACSW), pages 127–136. Australian Computer Society, Inc., jan 2005. ISBN 1-920-68226-0. URL http://dl.acm.org/citation.cfm?id=1082290.1082308. 1.3

[103] R Van Renesse, N Schiper, and F. B. Schneider. Vive la Diff\’erence: Paxos vs. Viewstamped Replication vs. Zab. IEEE Transactionson Dependable and Secure Computing, 12(4):472–484, 2015. URL http://arxiv.org/abs/1309.5671. 4.6

[104] Reiner Sailer, Xiaolan Zhang, Trent Jaeger, and Leendert van Doorn. Design and implementation of a TCG-based integrity measurementarchitecture. In Proceedings of the 13th USENIX Security Symposium, page 16, 2004. 1.3

[105] Tomas Sander and Christian F. Tschudin. Towards mobile cryptography. In Proceedings of the IEEE Symposium on Security and Privacy(S&P), pages 215–224, 1998. 1.3

[106] Tomas Sander and Christian F. Tschudin. Protecting Mobile Agents Against Malicious Hosts. Mobile Agents and Security (LNCS), 1419(1998):44–60, jan 1998. 1.3

[107] Nuno Santos, Krishna P. Gummadi, and Rodrigo Rodrigues. Towards trusted cloud computing. In Proceedings of the conference on Hottopics in cloud computing (HotCloud), page 3, jun 2009. 1.3

[108] Nuno Santos, Rodrigo Rodrigues, Krishna P. Gummadi, and Stefan Saroiu. Policy-sealed data: a new abstraction for building trustedcloud services. In Proceedings of the 21st USENIX Conference on Security Symposium, page 10, aug 2012. 1.3

[109] Joshua Schiffman, Thomas Moyer, Hayawardh Vijayakumar, Trent Jaeger, and Patrick McDaniel. Seeding clouds with trust anchors. InProceedings of the ACM workshop on Cloud computing security workshop (CCSW), page 43, New York, New York, USA, oct 2010. ACMPress. ISBN 9781450300896. doi: 10.1145/1866835.1866843. URL http://dl.acm.org/citation.cfm?id=1866835.1866843. 1.3

[110] F. B. Schneider. Implementing fault-tolerant services using the state machine approach: a Tutorial. ACM Computing Surveys (CSUR), 22(4):299–319, 1990. ISSN 0360-0300. URL http://portal.acm.org/citation.cfm?id=98167. 4.1, 4.2

[111] F Schuster and M Costa. VC3: Trustworthy data analytics in the cloud. In Proceedings of the IEEE Symposium on Research in Securityand Privacy (S&P), 2014. 1.3, 3.1, 3.3

[112] E. Shi, Adrian Perrig, and Leendert van Doorn. BIND: A Fine-Grained Attestation Service for Secure Distributed Systems. In Proceedingsof the IEEE Symposium on Security and Privacy (S&P), pages 154–168, may 2005. ISBN 0-7695-2339-0. doi: 10.1109/SP.2005.4. 1.3,2.2, 2.3

[113] R Sinha, S Rajamani, SA Seshia, and K Vaswani. Moat: Verifying Confidentiality of Enclave Programs. eecs.berkeley.edu. URLhttp://www.eecs.berkeley.edu/˜rsinha/research/pubs/ccs2015.pdf. 1.3

39

[114] Joseph G. Slember and Priya Narasimhan. Static analysis meets distributed fault-tolerance: enabling state-machine replication withnondeterminism. In Proceedings of the 2nd Conference on Hot topics in System Dependability (HotDep), page 2, nov 2006. 4.1

[115] Sean W. Smith and Steve Weingart. Building a high-performance, programmable secure coprocessor. Computer Networks, 31(9):831–860, 1999. ISSN 1389-1286. 2.4.1

[116] Joao Sousa, Edoardo Alchieri, and Alysson Bessani. State Machine Replication for the Masses with BFT-SMaRt. In Proceedings of theIEEE Conference on Dependable Systems & Networks (DSN), pages 355–362, 2014. 4.4.3, 4.5

[117] SQLite. http://www.sqlite.org/. 2.4.3

[118] Raoul Strackx and Frank Piessens. Fides. In Proceedings of the ACM Conference on Computer and Communications Security (CCS),page 2, oct 2012. 1.3

[119] G. Edward Suh, Dwaine Clarke, Blaise Gassend, Marten van Dijk, and Srinivas Devadas. AEGIS. In Proceedings of the 17th AnnualInternational Conference on Supercomputing (ICS), page 160, 2003. ISBN 1581137338. doi: 10.1145/782814.782838. 1.3, 2.3, 3.2

[120] Paul F. Syverson and Paul C. Van Oorschot. On Unifying Some Cryptographic Protocol Logics. In Proceedings of the IEEE Symposiumon Security and Privacy (S&P), page 14. IEEE Computer Society, may 1994. URL http://dl.acm.org/citation.cfm?id=882490.884204. 1.3, 2.5

[121] Roberto Tamassia and Nikos Triandopoulos. On the cost of authenticated data structures - Google Search. Technical report,2003. URL https://www.google.pt/search?q=On+the+cost+of+authenticated+data+structures&oq=On+the+cost+of+authenticated+data+structures&aqs=chrome..69i57.110j0j7&sourceid=chrome&es_sm=91&ie=UTF-8. 3.2

[122] Roberto Tamassia and Nikos Triandopoulos. Computational bounds on hierarchical data processing with applications to informationsecurity. In Proceedings of the 32nd International Conference on Automata, Languages and Programming (ICALP), volume 3580, pages153–165, Berlin,, jul 2005. 3.2

[123] Yuzhe Tang, Ting Wang, Ling Liu, Xin Hu, and Jiyong Jang. Lightweight authentication of freshness in outsourced key-value stores.In Proceedings of the 30th Annual Computer Security Applications Conference on - ACSAC ’14, pages 176–185. ACM Press, dec 2014.ISBN 9781450330053. doi: 10.1145/2664243.2664244. URL http://dl.acm.org/citation.cfm?id=2664243.2664244.3.2

[124] Trusted Computing Group. TPM 2.0. http://www.trustedcomputinggroup.org/resources/ tpm library specification. 1.3, 2.2

[125] Trusted Computing Group. MTM Specification v1.0 rev. 7.02, 2010. 2.3

[126] Trusted Computing Group. Mobile Trusted Module 2.0 Use Cases, 2011. 2.3

[127] Trusted Computing Group. TPM Main Specification Version 1.2, Rev. 116, 2011. 1.3, 1

[128] US CERT. Vulnerability Summary Bulletin. URL https://www.us-cert.gov/ncas/bulletins/SB14-237. 4.2

[129] Robbert van Renesse and Rachid Guerraoui. Replication techniques for availability. In Replication, pages 19–40. Springer-Verlag, jan2010. ISBN 3-642-11293-5, 978-3-642-11293-5. URL http://dl.acm.org/citation.cfm?id=2172338.2172340. 4.1

[130] Amit Vasudevan, Sagar Chaki, Limin Jia, Jonathan McCune, James Newsome, and Anupam Datta. Design, Implementation and Verifi-cation of an eXtensible and Modular Hypervisor Framework. In Proceedings of the IEEE Symposium on Security and Privacy (S&P),pages 430–444, may 2013. ISBN 978-0-7695-4977-4. doi: 10.1109/SP.2013.36. 2.2, 1, 3.3, 3.1, 4.4.3

[131] Bruno Vavala and Nuno Neves. Robust and Speculative Byzantine Randomized Consensus with Constant Time Complexity in NormalConditions. In Proceedings of the 31st Symposium on Reliable Distributed Systems (SRDS), pages 161–170, 2012. ISBN 978-1-4673-2397-0. doi: 10.1109/SRDS.2012.62. 4.2

[132] Bruno Vavala, Nuno Neves, and Peter Steenkiste. Securing Passive Replication Through Verification. In Proceedings of the 34st Sympo-sium on Reliable Distributed Systems (SRDS), 2015. 1.3

[133] Giuliana Santos Veronese, Miguel Correia, Alysson Bessani, Lau Cheuk Lung, and Paulo Verissimo. Efficient Byzantine Fault-Tolerance.IEEE Transactions on Computers, 62(1):16–30, jan 2013. ISSN 0018-9340. doi: 10.1109/TC.2011.221. 1.3, 4.1, 4.2, 4.3, 4.4, 4.4.2, 4.6

[134] J.H. Wensley, Leslie Lamport, J. Goldberg, M.W. Green, K.N. Levitt, P. M. Melliar-Smith, R.E. Shostak, and C.B. Weinstock. SIFT:Design and analysis of a fault-tolerant computer for aircraft control. Proceedings of the IEEE, 66(10):1240–1255, 1978. URL http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=1455383. 4.2

[135] Timothy Wood, Rahul Singh, Arun Venkataramani, Prashant Shenoy, and Emmanuel Cecchet. ZZ and the art of practical BFT execution.In Proceedings of the 6th Conference on Computer Systems (EuroSys), pages 123–138, 2011. ISBN 9781450306348. doi: 10.1145/1966445.1966457. URL http://dl.acm.org/citation.cfm?id=1966445.1966457. 4.1, 4.3, 4.4

[136] Hsin-Jung Yang, Victor Costan, Nickolai Zeldovich, and Srinivas Devadas. Authenticated storage using small trusted hardware. InProceedings of the ACM workshop on Cloud computing security workshop (CCSW), pages 35–46, nov 2013. 3.2

[137] Andrew C. Yao. Protocols for secure computations. In Proceedings of the 23rd Annual Symposium on Foundations of Computer Science(SCFS), pages 160–160–164–164, nov 1982. 1

[138] Jian Yin, Jean-Philippe Martin, Arun Venkataramani, Lorenzo Alvisi, and Mike Dahlin. Separating agreement from execution for byzan-tine fault tolerant services. In Proceedings of the Symposium on Operating Systems Principles (SOSP), volume 37, 2003. 4.4

[139] ZeroMQ. http://zeromq.org/. 4.4.3

40