Design, Implementation and Evaluation of Security iniSCSI-based Network Storage Systems

Shiva ChaitanyaComputer Science and

EngineeringPennsylvania State University

chaitany@cse.psu.edu

Kevin ButlerComputer Science and

EngineeringPennsylvania State University

butler@cse.psu.edu

Anand SivasubramaniamComputer Science and

EngineeringPennsylvania State University

anand@cse.psu.edu

Patrick McDanielComputer Science and

EngineeringPennsylvania State University

mcdaniel@cse.psu.edu

Murali VilayannurMathematics and Computer

Science DivisionArgonne National Laboratory

vilayann@mcs.anl.gov

ABSTRACTThis paper studies the performance and security aspectsof the iSCSI protocol in a network storage based system.Ethernet speeds have been improving rapidly and networkthroughput is no longer considered a bottleneck when com-pared to Fibre-channel based storage area networks. How-ever, when security of the data traffic is taken into consid-eration, existing protocols like IPSec prove to be a majorhindrance to the overall throughput. In this paper, we eval-uate the performance of iSCSI when deployed over standardsecurity protocols and suggest lazy crypto approaches to al-leviate the processing needs at the server. The testbed con-sists of a cluster of Linux machines directly connected tothe server through a Gigabit Ethernet network. Micro andapplication benchmarks like BTIO and dbench were used toanalyze the performance and scalability of the different ap-proaches. Our proposed lazy approaches improved through-put by as much as 46% for microbenchmarks and 30% forapplication benchmarks in comparison to the IPSec basedapproaches.

Categories and Subject DescriptorsD.4 [Operating Systems]: Storage Management; D.4.6[Security and Protection]: Cryptographic controls

General TermsSecurity, Design, Performance, Experimentation, Measure-ment

KeywordsiSCSI, IPSec, Encryption, Authentication

Copyright 2006 Association for Computing Machinery. ACM acknowledgesthat this contribution was authored or co-authored by an employee, contrac-tor or affiliate of the U.S. Government. As such, the Government retainsa nonexclusive, royalty-free right to publish or reproduce this article, or toallow others to do so, for Government purposes only.StorageSS’06, October 30, 2006, Alexandria, Virginia, USA.Copyright 2006 ACM 1-59593-552-5/06/0010 ...$5.00.

1. INTRODUCTIONData storage is an increasingly important asset in to-

day’s fast growing data-intensive network services. Giventhe overwhelming and mature Internet infrastructure, IP-based protocols for storage networking are beginning to emerge.The iSCSI protocol specifies how to access SCSI devices overa TCP network. The protocol encapsulates SCSI disk accessrequests (in the form of SCSI commands, data blocks andcommands) into TCP packets and transmits them over IPnetworks. Although, implementing storage over traditionalIP networks is economical and convenient, performance andsecurity of the iSCSI protocol stack are equally important is-sues that need to be dealt with effectively. In this paper, westudy the interplay of security and performance in the iSCSIprotocol stack in a clustered network storage environment.

Ethernet speeds have been improving rapidly (multi-gigabitper second) so much so, that network throughput is nolonger considered a bottleneck when compared to Fibre-channel based storage area networks. However, when secu-rity of the data traffic is taken into consideration, existingsecure protocols that operate at the lower layers of the stacklike IPSec prove to be detrimental to the overall through-put. On a typical IPSec implementation, we have measureda point-to-point throughput reduction of as much as 75%.Further, this throughput reduction is even more significantwhen multiple clients are active, resulting in poor scalabil-ity in a shared environment. The primary reason for thisdrastic reduction in throughput is the overhead of IPSecprocessing of all client connections and data at the centralserver. In this paper, we characterize the performance of theiSCSI protocol stack when used in conjunction with IPSec,demonstrate the critical path overheads of protocol process-ing and present the design and experimental results usingtechniques that reduce protocol overheads while still pre-serving security guarantees.

iSCSI-based storage systems are vulnerable to security at-tacks from various sources. Since they rely on TCP/IP basednetworks for data transmission, all the associated problemswith these kinds of networks are inherited by the iSCSI pro-tocol. iSCSI data that is sent in clear across the publicnetworks is exposed to passive and active attacks by an

adversary snooping on the wire. Passive attacks involvegathering of information by eavesdropping on the physicalmedium and using the obtained knowledge to cause damageat a later point in time. This type of scenario is possiblein situations where mission critical data of an organizationtraverses public networks. Leakage of such information, byitself, constitutes a major security breach. Active attacksinvolve more real-time intrusion by either modifying thepacket contents, replacing them by new packets, denial-of-service attacks, etc. It also includes man-in-the-middle typewhere the adversary deceives the sender and receiver intobelieving that they are communicating with each other, butin reality, all messages are intercepted and modified appro-priately for either end of the link.

Attacks in collusion with the iSCSI server are also possiblewhere the adversary can simply obtain access to the criticaldata stored on the target disks. These kinds of attacks byan untrusted server can be prevented only by storing dataencrypted on disk and relying on some trusted third partyto do the management and secure distribution of encryptionkeys. The trusted entity provides the services for the clientsand controls access to the server based on capabilities is-sued apriori. This scheme again raises the issue of placingthe trust on a central authority that can be compromised infuture. Analysis of this kind of problem usually results intrade-offs between developing a secure central solution and adistributed solution with key management among peer ma-chines. Distributed solutions need to be based on a robustalgorithm that remains foolproof against compromise of oneor few number of clients. A large number of clients colludingand launching attacks is tough to protect against and is ingeneral not considered a viable problem.

In this work, we only address security issues relating tothe data traffic and make an assumption that the users ofthe storage network and servers are trusted parties. Some ofthe popular techniques like LUN Masking and Zoning thatare employed in Fibre-Channel SANs can also be appliedin iSCSI networks to guard against unfair usage of storageeven by trusted clients. LUN Masking is a mechanism thatallows nodes on a SAN to only see the LUNs that they areauthorized to access. It works by mapping the LUNs toWWNs (World Wide Name used in FC networks, analogousto IP address) and only providing access to certain LUNsfrom certain WWNs. Zoning is another method where re-strictions are placed on the communication between certainsets of WWNs. The access control is implemented eitherin the hardware driver on the initiators or incorporated onthe Fibre-Channel switches. In our work, we assume thatthe initiators are trusted to perform fair accesses and hencethis issue is not considered further. Also, in the normalscenario the initiators of a iSCSI network are applicationservers running in an organization and are typically trustednot to perform such illegal accesses.

The rest of this paper is organized as follows. The nextsection describes the IPSec protocol in greater detail, char-acterizes the performance overheads of the stack and moti-vates the need for lazy crypto techniques to ameliorate theperformance drop. Section 3 outlines the design of the lazydecryption and authentications schemes. Implementationissues and performance evaluation with micro-benchmarksand real applications are presented in Section 4. We dis-cuss related work in Section 5 before concluding the paperin Section 6.

2. ISCSI AND IPSECThe iSCSI protocol definition provisions that the end points

of communication be authenticated before starting an iSCSIsession. An initiator must first establish a session with thetarget before obtaining access to storage data. A session iscomposed of one or more TCP connections. When the lead-ing TCP connection of a session is setup, the login phasebegins with exchange of authentication information betweenthe involved parties through mechanisms such as the Chal-lenge Handshake Authentication Protocol (CHAP) and theSecure Remote Password (SRP) protocol. After the ses-sion’s operational parameters are negotiated, the full datatransmission phase begins.

While the login process provides mutual authentication ofthe endpoints before the initiation of a session, iSCSI doesnot account for per-packet authentication and integrity ofdata during the full featured phase. In addition, privacy ofthe confidential data sent across the vulnerable IP networksis also not taken into consideration. One of the standardsolutions is to run iSCSI in conjunction with some securityscheme available in a lower layer of the protocol stack suchas IPSec.

IPSec, an extension to IP, is a security protocol that canbe established between two machines in order to secure datatraffic. It provides confidentiality, authentication and in-tegrity services to upper layers of the network stack. IPSecis used to secure tunnels against false data origins and en-crypts traffic to combat active and passive intruders whocan listen or modify the traffic. It consists of two separateprotocols, IP Authentication Header (IPSec AH) and IP En-capsulating Security Payload (IPSec ESP).

IPSec AH provides connectionless integrity and data ori-gin authentication, while IPSec ESP provides both thosefeatures and confidentiality. The Authentication Data fieldis computed by performing a one-way hash function (e.g.MD5, SHA-1) on the payload that includes all the data bitsfrom the upper layers like TCP/UDP and network applica-tions. The common hash function is used again at the re-ceiver to compute the authentication data from the payloadand is checked against the one provided in the AH header.Any modification to an original IP packet or a newly cre-ated one from an unauthorized host can be detected at thereceiver. Figure 1 (a) shows the IP AH header format.

IPSec ESP provides confidentiality in addition to connec-tionless integrity and data origin authentication. The pay-load data field is computed by applying a symmetric en-cryption algorithm (e.g. Data Encryption Standard DES,Advanced Encryption Standard AES) to the original datacomprising of TCP or UDP data. The encrypted data isfollowed by the authentication data computed in the samemanner as IPSec AH. Figure 1 (b) depicts the format of theIPSec ESP header.

There are two ways by which the choice of cryptographicalgorithms and the symmetric keys used by IPSec can be seton the two communicating machines. Manual configurationof the algorithms and setting the pre-shared keys is doneby including the information in a system configuration file.Another option is to use the IKE (Internet Key Exchange)protocol. IKE works by allowing IPSec-capable devices toexchange security associations (SAs) to populate their se-curity association databases (SADs). A Security Associa-tion (SA) is a set of security information that describes aparticular kind of secure connection between two machines.

(a) (b)

Reserved

Sequence Number

Authentication Data

Security Parameter Index (SPI)

Next Header Payload Length

Authentication Data

Padding Length Next Header

Padding

Payload Data

Sequence Number

Security Parameter Index (SPI)

Figure 1: (a) IPSec AH Format, (b) IPSec ESP For-mat

It contains information like IP addresses, security protocolused, cryptographic algorithm, keys, etc. IKE also providesthe benefits of dynamic SA establishment and dynamic re-keying.

2.1 IPSec performanceEnsuring high performance of the IPSec protocol is crucial

to obtaining good throughput from the IPSec-based iSCSIstack. With the increase in available network throughput,thanks to the deployment of multi-gigabit Ethernet, the fo-cus has shifted from the bandwidth offered by the phys-ical medium to the network stack (CPU processing over-heads) on either side of the link. Authors in [8] identify thetime spent in TCP/IP processing and interrupt overheadsas the two major bottlenecks in obtaining high iSCSI per-formance. With the inclusion of IPSec processing to thenetwork flow, there is an additional slowdown to the over-all throughput. Further, processes of encryption/decryptionare highly CPU-intensive and affect other applications run-ning on the system. One of the major factors hampering theperformance of IPSec is the very fact that it is implementedat the IP layer (lower layer of the stack). Thus, application-level information and requirements cannot be exploited bythe IPSec layer.

Let’s take a closer look at the transmission of a singleIP packet through the IPSec layer on the sender side. Thefollowing description is specific to the native IPSec imple-mentation in Linux kernel 2.6. Though the exact processingtimes and overheads may vary slightly among different im-plementations of IPSec, all of them follow a similar set ofpacket processing rules. IPSec processes a packet based onIPSec Security Policy (SP) and IPSec Security Association(SA). SP indicates if a packet should be processed as anIPSec packet or dropped. The chosen packets are processedby the IPSec stack with some parameters included by SA.IPSec Security Policy is checked next to determine if thepacket is required to be processed by IPSec. Then, for ev-ery IPSec packet, the SA database is queried to obtain thedestination chain of output functions. The output functionsperform cryptographic manipulations on the packet basedon the type of the protocol used, AH or ESP. Note thatthere can be more than a single output function per packetif multiple security protocols are configured for a particularroute. Finally, the IPSec packet is passed to the destinationchain.

We conducted a small set of experiments to study the per-formance (throughput) of IPSec AH and ESP for the ttcp

program for a single client accessing a server. The senderprocesses continuously transmit data of a fixed size to thereceiver which is listening on a TCP socket. The average

0.5 8 640

20

40

60

80

100

120

Packet Size in KB

Net

wor

k th

roug

hput

in M

B/s

No IPSec

IPSec AH − HMAC md5

IPSec AH − HMAC SHA1

IPSec ESP − DES + HMAC md5

IPSec ESP − DES + HMAC SHA−1

IPSec ESP − 3DES + HMAC md5

IPSec ESP − 3DES + HMAC SHA−1

IPSec ESP − BlowFish192 + HMAC md5

IPSec ESP − BlowFish192 + HMAC SHA−1

IPSec ESP − AES192 + HMAC md5

IPSec ESP − AES192 + HMAC SHA−1

Figure 2: Maximum Throughput of IPSec for differ-ent packet sizes

throughput and latency for each packet size was computed.This experiment was repeated for a setup without IPSec(No-IPSec), IPSec with AH (IPSec-AH) and IPSec withESP (IPSec-ESP). Measurements with IPSec AH and IPSecESP were repeated for different algorithms supported in theLinux kernel 2.6 IPSec implementation. The test machines,each a 2-processor Intel(R) Xeon(TM) 3.06GHz (with hyper-threading turned on) running a 2.6.13 Fedora 4 kernel weredirectly connected via a Gigabit Ethernet Switch. The net-work interface cards on these machines were Intel 82546EBGigabit Ethernet adapters. Figure 2 shows the maximumnetwork throughput obtained by running ttcp on top ofIPv4, IPSec (AH and ESP). Different algorithms were cho-sen for the two IPSec protocols. The maximum throughputattainable was determined by increasing the number of con-current threads in the program till the host CPU utilizationreaches 100% or network becomes saturated. For IPv4, net-work bandwidth was fully utilized for ≥ 8KB packet sizes.For a packet size of 512 bytes, the maximum throughputobtained was ' 75 MB/s. For IPSec AH, the maximumthroughput values obtained for packet sizes of 512 bytes, 8KB and 64 KB was 22, 47, 62 MB/s respectively. Comparedto IPv4, that amounts to a throughput reduction of 70%,58% and 45% respectively. For IPSec ESP, the throughputreductions were more drastic with values of 77%, 74% and71%. The point to point throughput suffers as well whenmultiple clients are active thus resulting in poor scalabilityin a shared environment. The primary reason for this degra-dation is the centralized bottleneck caused by the IPSec pro-cessing of all the client data at the server. This type ofperformance degradation of IPSec when running iSCSI canhave a detrimental affect on the overall throughput of theI/O applications. In the critical path of IPSec processing,there are a pair of crypto processes (encryption/decryptionand/or HMAC) that run on two machines. The next fewsections describe how these overheads can be amelioratedwhile still preserving the security guarantees.

3. PROPOSED SOLUTIONSConsider a typical scenario in an iSCSI environment, that

comprises of a set of iSCSI clients called the initiators andan iSCSI target that is attached to the storage disks. Thesedisks are exported at the block-level to the clients so thatthey can store huge quantities of data and retrieve them

Application

Application

Application

SCSI

iSCSI

TCP

IP

NIC Driver

IPSec

SCSI

iSCSI

TCP

IP

NIC

Driver

IPSec

NIC Driver

IP IPSec

TCP

iSCSI

SCSI

Initiator 1

Initiator 2

iSCSI write Data

Target

iSCSI Read Data

(Data Encryption/Authentication with key A)

(Data Decryption/Authentication with key A)

(Data Decryption/Authentication with key B)

(Data Encryption/Authentication with key B)

Figure 3: Path of data packets during a Write by Ini-tiator 1 and a subsequent Read by Initiator 2. KeysA and B are IPSec symmetric keys of Initiator1-Target and Initiator2-Target respectively

when necessary. The storage management system at theiSCSI target takes care of all requirements like availability,maintenance, backup, disk space management etc. When aniSCSI initiator writes a data block to the target, it is storedon the disk as it is received. If IPSec was used to securethe network between the initiator and target, the data sentfrom the initiator is transformed using the symmetric IPSeckeys and back again at the target before it reaches the iSCSIlayer. This data when requested by the same client or someother client is manipulated again at the IPSec layer on theserver.

Figure 3 depicts this flow of read-write requests for IPSecESP and it is apparent that the crypto processing of thestorage data blocks that take place at the server is superflu-ous and could be done away with if the same key is used for adata block throughout the entire path. In other words, sincethe server simply acts as a storage reservoir and doesn’t usethe data itself, it need not be involved in the data processingif the transformation keys are securely communicated to theclients. Note that the crypto processing of the control data(iSCSI headers) cannot be avoided at the server and needsto be performed in the critical path. Correct functioningof the iSCSI protocol requires the control data be immedi-ately acted upon at the server. Since IPSec is implementedat the IP layer it is not possible to selectively act on cer-tain portions of the IP payload (TCP header, iSCSI header)and leave the iSCSI data portion untouched. IPSec is de-signed to provide application independent security servicesand is oblivious of upper level characteristics and require-ments. Consequently, crypto operations are performed inan identical manner on ALL traffic between two IP nodes.Thus our observation to improving performance is to movethese operations to the iSCSI layer by which it is possibleto reduce unnecessary processing from the critical path. Wenow explain the first of our two mechanisms namely LazyDecryption.

3.1 Lazy DecryptioniSCSI defines its own packets that are referred to as iSCSI

Protocol Data Units (PDUs). A iSCSI PDU consists of aheader and possible data, where the data length is specifiedwithin the iSCSI PDU header. The basic segment headersize is fixed and is 48 bytes. Additional headers can beattached to the basic segment and this information is spec-ified in the TotalAHSLength field (see Figure 4). As rea-soned above, when used in conjunction with IPSec ESP, thedecryption of the iSCSI data is not really needed at thetarget for writes. Similarly during reads, data is alreadyin encrypted form at the server and can be sent across thenetwork without any transformations. iSCSI PDU headercontains important control information like the iSCSI ses-sion id, SCSI command and other operational parametersthat need to be protected from external attacks as much asthe critical data. Delayed decryption of the header at theserver does not work because the iSCSI target layer needsto process all the fields in it for proper functioning of theprotocol.

In our lazy decryption approach, we aim to provide theconfidentiality services to iSCSI data. IPSec AH can be usedfor providing iSCSI data origin authentication and its in-tegrity. In addition, the IPSec Security Association Databasewill be populated with a IPSec ESP encryption algorithmand a symmetric key. The key is either pre-shared or IKE isused to dynamically establish and update it. However, theSecurity Policy on the two machines is not configured to useIPSec ESP. This allows for symmetric encryption key man-agement between the two machines without actually usingit for encrypting data traffic at the IPSec layer. This sym-metric encryption key is used to encrypt the iSCSI PDUheader at the iSCSI layer. The symmetric encryption algo-rithm used by the iSCSI initiator and target can either behard coded in the implementation or negotiated as one ofthe operational parameters during the login phase. In thecurrent work, we assume that they are set manually in theiSCSI configuration files. The novelty of our approach lies inthe fact that we exploit the secret key of IPSec and use it tocommunicate between the machines at the application layer.The following subsections explain in detail how lazy decryp-tion is incorporated at the iSCSI layers on the initiator andtarget software.

3.1.1 At the Initiator

For iSCSI operations that require only the transmissionof the iSCSI PDU header (e.g SCSI READ Command), im-plementation of lazy decryption is straightforward. Thesymmetric encryption key obtained from IPSec is used tofirst encrypt the basic header (48 bytes). The set of alladditional headers is encrypted as a single unit using thesame key. This is followed by the transmission of the entireheader. For operations that require subsequent transmis-sion of data blocks following the PDU header (e.g. SCSIWRITE), things are a little more complicated. We cannotuse the symmetric encryption key to encrypt the data blocksas well because decryption of the shared data at some otherclient would require knowledge of the secret key known onlyto the original initiator and the target. We overcome thisby generating a random Block Encryption Key (BEK) forevery data block about to be written to the server. TheseBEKs are used to encrypt the storage blocks in the iSCSIPDU data and are stored sequentially in a new Additional

Header Segment (AHS) appended to the end of iSCSI PDUheader. Note that the BEKs are not one-time keys and per-sist only till some initiator generates a new key during thenext write to the same block. Therefore, the BEKs need notbe remembered by the client and can be disposed of. Fig-ure 4 shows the modified iSCSI PDU header format used forthis scheme. The confidentiality of the iSCSI PDU header(including the BEKs) is protected by the symmetric IPSecencryption key and that of the iSCSI PDU data is protectedby the BEKs. This is an application of skip encryption [13]used in security schemes. When the initiator receives iSCSIPDUs from the target, it first decrypts the 48 byte headerusing the symmetric key and checks to see if there are addi-tional headers and data segment. Additional headers are de-crypted in the same manner using the symmetric key. Datablocks, if present are decrypted using the BEKs sent alongin the last AHS.

3.1.2 At the Target

The implementation at the iSCSI target layer is quite sim-ilar to that of the initiator but with two important differ-ences. The Block Encryption Keys received are not usedto decrypt the data blocks. Rather, the encrypted dataare stored directly on the iSCSI exported disks. Separatestorage space, different from the iSCSI disks, is needed toaccumulate the BEKs. The keys can either be located atfixed positions on a disk or stored in a file system. In thecurrent implementation, we chose to use file storage for thekeys and made sure that the ones belonging to adjacent log-ical block addresses are co-located in a file. This improvesthe access time of the keys during iSCSI data transfers. Wealso implemented a simple cache to store frequently accessedBEKs in memory.

Now we describe our second mechanism called Lazy Au-thentication that reduces the CPU load at the iSCSI serverby delegating data integrity checks to the clients.

3.2 Lazy AuthenticationIn this approach, we intend to achieve origin authentica-

tion and integrity of data blocks by performing the checksonly at the clients. IPSec key management service is usedto share a symmetric key between a client and the server.Hashed message authenticated code (HMAC) of the iSCSIheader is generated at the sender and checked immediatelyat the receiver to detect any modifications to control data.However, the server does not perform any HMACs on thedata blocks. The following subsections describe the methodin detail as implemented on the initiator and target ends.

3.2.1 At the Initiator

For operations involving transmission of only the iSCSIheader, the initiator generates a HMAC for the entire header(including the Additional header segment) and appends itat the end. Typically the size of the generated output isabout 16 bytes (HMAC-MD5) or 20 bytes (HMAC-SHA1).The symmetric HMAC key used is the one obtained from theIPSec layer. If data blocks are transmitted along with theheader, then a random Block Authentication Key (BAK)

is generated for every block and is used to generate the blockHMAC. The BAKs are transmitted in encrypted form (usingthe symmetric IPSec key) and constitute a part of the iSCSIheader. Figure 5 shows the modified iSCSI header to sup-port lazy authentication. The block HMACs are appended

. | I | Opcode F | Opcode−specific fields

TotalAHSLen

0

DataSegmentLength

( optional)

Basic HeaderSegment (BHS)

Last AHS

( optional)

( optional)

4

8

12

16

48

Block Encryption Key 1

Block Encryption Key m

LUN or Opcode−specific fields

Opcode−specific fields

Initiator Task Tag

Additional Header Segment 1 (AHS)

Additional Header Segment n−1 (AHS)

Header−Digest ( optional)

Figure 4: Modified Format of iSCSI PDU header forLazy Decryption. iSCSI PDU header is encryptedusing the IPSec symmetric encryption key and theData Segment is protected using the BEKs.

. | I | Opcode F | Opcode−specific fields

TotalAHSLen

0

DataSegmentLength

( optional)

Basic HeaderSegment (BHS)

Last AHS

( optional)

( optional)

4

8

12

16

48Additional Header Segment 1 (AHS)

Additional Header Segment n−1 (AHS)

LUN or Opcode−specific fields

Opcode−specific fields

Initiator Task Tag

Block Authentication Key (Encrypted) m

Block Authentication Key (Encrypted) 1

Hashed Code (HMAC) of Header

Figure 5: Modified Format of iSCSI PDU headerfor Lazy Authentication. The Block Authenticationkeys are encrypted using the secret IPSec key andsent in the AHS. They are followed by the HMACof the header.

Table 1: Crypto transformations of iSCSI PDUs. (H) denotes Header of iSCSI PDU, (B) denotes Body/datasegment of the iSCSI PDU, (Ck) is the secret communication key (symmetric IPSec key ) and (Bk) is therandom body key (BAK or BEK). E and D denote encryption and decryption respectively.

Scheme iSCSI PDU on network ClientWrite ClientRead ServerWrite ServerRead

Plain iSCSI M = H + B

IPSec AH M = (H + B), HMACCk(H + B) HMACCk(M) HMACCk(M) HMACCk(M) HMACCk(M)IPSec ESP eM = ECk(H + B) ECk(M), HMACCk(M), ECk(M), HMACCk(M),

M = eM, HMACCk(eM) HMACCk(M) DCk(M) HMACCk(M) DCk(M)Lazy Auth. M = (H + B), HMACCk(H), HMACCk(H), HMACCk(H), HMACCk(H) HMACCk(H)

HMACBk(B) HMACBk(B) HMACBk(B)IPSec AH eH = ECK (H), eB = EBk(B) HMACCk(M), HMACCk(M), HMACCk(M) HMACCk(M)+ Lazy Dec. M = (eH + eB), HMACCk(eH + eB) EBk(B) DBk(eB)Lazy Auth eH = ECK (H), eB = EBk(B) HMACCk(eH), HMACCk(eH), HMACCk(eH) HMACCk(eH)+ Lazy Dec. M = (eH + eB), HMACCk(eH), HMACBk(eB) HMACBk(eB)

HMACBk(eB) EBk(B) DBk(eB)

to the end of the iSCSI PDU data segment. Since the blockHMACs are created using the encrypted BAKs present inthe header, they need not be associated with the headercontent. Attacks where the data blocks and their HMACsare replaced on the wire with previously seen ones can bedetected since different BAKs are used for different blocksand the adversary does not know them. When receiving aheader, the client computes its HMAC and checks to see if itmatches with the one provided in the header itself. This en-sures that the header originated from the iSCSI server andwas unmodified in transit. If data blocks are also receivedalong with the header, the BAKs are first decrypted andused to compute the block HMACs which are then checkedagainst the ones received with the iSCSI PDU.

3.2.2 At the Target

The iSCSI header received is immediately authenticated(origin and integrity) using the shared IPSec HMAC key.Data blocks if received, are not checked for their authen-ticity at the server. The BAKs are however decrypted andstored on a separate file system in the same manner as de-scribed in the lazy encryption scheme. Since the authentic-ity of the data blocks is not checked until a subsequent readby a client, they cannot overwrite existing data on the disk.Therefore, there is a need for creating versions of unauthen-ticated data blocks till a client confirms the successful com-pletion of a read. We implement block versioning by storinga sequence of unauthenticated blocks on separate storage.When a client performs a read, it receives the latest ver-sion from the server and if the received blocks cannot beauthenticated, the read is re-tried with the previous versionand so on. A variant of this technique was proposed in [14,20] in the context of a fault tolerant Byzantine distributedstorage system. The successful completion of a read by aclient enables the server to commit the correct version on tothe iSCSI disk and the previous and incorrect versions aredestroyed.

The lazy approaches help in decreasing the CPU load atthe server by reducing the crypto processing of the datablocks at the server. Table 1 shows the overall picture ofthe crypto manipulations of the iSCSI header and data atthe client and server for reads and writes. The first columnenumerates different schemes that differ in the manner inwhich the iSCSI PDUs are handled from the crypto perspec-tive. The second column shows the format of the messageson the network. The last four columns show the actual in-bound and outbound crypto processing on the clients and

the server. Note that the crypto processing at the serverend is significantly reduced in the lazy approaches.

4. IMPLEMENTATION ISSUESFor implementing the crypto operations at the iSCSI layer,

we used the cryptographic API available in 2.6 Linux ker-nels. The scatter list based API takes page vectors as argu-ments and works directly on pages. The API supports threetypes of transforms, ciphers (Encryption), Digests (Hashing)and Compression. Since iSCSI software drivers are imple-mented as kernel modules, the API (kernel methods) wasintegrated directly without any overhead of library functioncalls. The XFRM interface was used to retrieve the sym-metric encryption key from the IPSec layer’s SAD.

If the basic unit of the crypto operations were a singleblock, reads and writes simply involve generating HMACsand encrypted blocks and transmitting them directly to theserver. However, this requires a stream of kernel functioncalls for every data block. To reduce this overhead, a fixedsize of consecutive blocks can be chosen as a block set andcrypto operations can be performed at the granularity of ablock set. However, block sets introduce complications whena subset of blocks are accessed. Suppose a set of consecutiveblocks A, B, C and D form a single encryption unit. Everytime a subset of these blocks (say B,C) is written by aniSCSI initiator, the remaining blocks in the set (A,D) mustfirst be retrieved from the target and decrypted at the client,following which all the blocks in the set must be re-encryptedwith the new key and sent back to the server. In the currentwork, we use a block as the granularity of crypto operationsto avoid this problem.

One of the tunable parameters in the above implementa-tion of lazy approaches is the mapping between Block Keys(BEK and BAK) and data blocks. A unique Block Keycan be associated with a single block or multiple blocks. Ifthe mapping were one-to-one, then the total number of keysneeded for a iSCSI disk is the capacity of the disk in blocks.For a disk capacity of 36GB and key size of 16 bytes, thestorage space required for Block Keys alone at the iSCSIserver would be (36/512)*16 ' 1.1 GB, assuming that thetypical size of a block is 512 bytes. One-to-many mappingsdecrease the amount of storage needed for the keys but in-crease the security vulnerabilities of the data blocks.

In the lazy authentication approach described in the Sec-tion 3.2, versions need to be maintained for unauthenticateddata blocks. The number of versions for each block increaseswith every consecutive write. This can cause an explosion

of storage requirements at the server for some applications.To avoid this, periodic garbage collection can be performedat the server that simply authenticates some versions of ablock and commits the latest valid block.

With increasing Ethernet speeds, CPU overheads of pro-tocol processing is a cause for concern in servers. By remov-ing the encryption and decryption processing needs at theiSCSI server as shown in the lazy decryption and authenti-cation schemes, a significant bottleneck has been removed.This not only increases the throughput of a single initiator totarget, but also makes the system much more scalable withincrease in number of simultaneous initiators. Of course,there is a new overhead of storage and retrieval of BlockKeys but we estimate that this is much less compared tothe CPU-intensive crypto operations and corroborated byour results. As mentioned before, one of our major focus inthe current work is the performance of iSCSI protocol whendeployed in a shared storage environment with potentiallymultiple initiators accessing the same blocks simultaneouslyfrom a trusted server. With this setup, key storage andmanagement at the server provides an efficient mechanismto perform secure data block transfer.

4.1 Evaluation SpaceIn a TCP/IP network, the underlying physical medium

can vary considerably. Some networks already support phys-ical security while others may not have any such mecha-nisms. Also, the security requirements of applications run-ning on top on iSCSI can differ significantly as well. We clas-sify the security needed by iSCSI depending on the existingunderlying infrastructure and requirements of applicationsinto three levels. The first level is No-Security. Some iSCSIsystems are deployed in controlled environments where thenodes operate under physically secure conditions and no ad-ditional security guarantees need to be provided. This is alsothe case in situations where the underlying network has al-ready been provided with security channels like VPN. Wename this level as Security-0.

The second level of security offers origin authenticationand integrity of iSCSI data. This is applicable in situationswhere active attacks are possible by snooping on the pub-lic networks. Confidentiality of data may be unimportantin web servers where the content is readable by anybodybut the users require the guarantee that the data did arrivefrom the server and that it was not modified on the way.Also, the server needs to assured of the identity of the userand integrity of user data received. Sometimes, even in a pri-vate network where the privacy of data is preserved, the endusers at the periphery of the network still need to be authen-ticated. The security level, named Security-1 addresses allthese issues by providing authentication and data integrity.One might argue that the IPSec AH and Lazy Authenti-cation do not exactly provide the same security gauranteesbecause the TCP and IP headers are not covered in the lat-ter. However the spoof attacks potentially caused by this inLazy Authentication can be detected easily because the twocommunicating parties share a IPSec symmetric key and useit to authenticate the iSCSI header. The provision againstanti-replay attacks by IPSec AH which uses a monotonicallyincreasing sequence number for the purpose is also present inlazy authentication with the initiator task tag in the iSCSIPDU header taking up the role of the nonce.

The third level provides the services of Security-1 along

Table 2: Three levels of iSCSI security and theirimplementation methods

Name Services Implementation

Offered Choices

Security-0 No security S0 - No changes to iSCSI

Security-1Data origin S1-1 - IPSec AH (Soft)authenticationand Integrity S1-2 - Lazy Auth (Soft)

Security-2

S2-1 - IPSec ESP (Soft)S2-2 - IPSec ESP (Hard)

Data origin S2-3 - Lazy Decryption (Soft)authentication,

+IPSec AH (Soft)Integrity and S2-4 - Lazy Decryption (Hard)Confidentiality

+IPSec AH (Soft)S2-5 - Lazy Decryption (Soft)

+Lazy Auth (Soft)S2-6 - Lazy Decryption (Hard)

+Lazy Auth(Soft)

with data confidentiality. This level of guarantee is neededin situations where the end hosts are communicating in acompletely insecure underlying network. TCP/IP runningon public networks comes under this category. SensitiveiSCSI data traversing such paths are susceptible to all typesof security attacks discussed in this work. By providingorigin authentication, data integrity and encryption, thislevel Security-2 guards against the vulnerabilities exposedto iSCSI data traffic.

Table 2 summarizes the three security levels. The thirdcolumn in the table describes the different implementationchoices that we have at our disposal to satisfy the require-ments of the three iSCSI security levels. The labels SN-Mindicates the design choice with N representing the securitylevel and M being the index within the group.

For Security-0, we use the software implementations ofiSCSI initiator and target in their original form. Both ofthem are configured to run on a traditional TCP/IP stackwith no added functionality. Security-1 is provided by run-ning iSCSI on top of IPSec AH or using Lazy Authentica-tion in software. To avoid design space explosion, we re-stricted ourselves to using hmac-md5 uniformly across allimplementations of the one-way cryptographic hash func-tion and aes192-cbc as the encryption algorithm. Six dif-ferent implementations were chosen for Security-2. Thefirst two are iSCSI running over IPSec ESP implementedin software or the encryption/decryption offloaded to hard-ware. The details of the crypto hardware used in our workare presented in the next section. The next two involve thecombination of Lazy Decryption and IPSec AH with the en-cryption/decryption implemented in software or hardware.Similarly the last two are using the combination of LazyDecryption and Lazy Authentication.

4.2 Experimental TestbedOur iSCSI testbed consists of x86 based SMP machines.

Each client machine is equipped with dual Hyper-threadedIntel Xeon processors clocked at 3.06 GHz equipped with 1GB RAM. All the nodes are connected directly via a Giga-bit Ethernet network with Intel 82546EB Ethernet adapters.The iSCSI server is equipped with dual AMD Opteron pro-cessors clocked at 1.6 GHz with 1 GB RAM. The serveris connected to a nStor Fibre-Channel storage array con-taining 12 Seagate 36GB drives of 15K RPM speed. Theexternal transfer rate of each disk is 200 MB/s, Average la-

tency is 2 msec and Average Read/Write seek time is 3.5/4.0msec. The operating system running on the client machinesis Fedora Core 4 with a kernel version 2.6.13-1.1532, whilethe server uses Fedora Core 2 with kernel version 2.6.10-1.771. Open source iSCSI initiator open-iscsi-0.4-408 [2]and Ardis target implementation iscsitarget-0.4.11 [1]were used for the software versions of iSCSI initiator andtarget. In order to run our applications, we use Redhat’sopen source clustering SAN file system (GFS version 6.1)which allows applications to simultaneously read and writeto a shared file system on the SAN.

For hardware acceleration of crypto functions, we used thePowerCrypt 5x PCI card. It supports encryption algorithmssuch as AES, DES, 3DES and RC4. Based on Hifn’s 7956 en-cryption processor, PowerCrypt 5x can provide AES cipherspeeds of approximately 80 MB/s. The card is supported onLinux systems through OCF (Open Cryptographic Frame-work) drivers. The drivers support hardware acceleration ofgeneral purpose crypto functions. We instrumented the 2.6kernel to incorporate OCF asynchronous offloading of ESPprocessing for the native IPSec stack.

4.3 Benchmarks UsedTo evaluate the performance of the iSCSI system with

security overheads, we used a set of micro-benchmarks andapplications. Rawio is a workload generator that is usedto extensively evaluate I/O subsystems. The applicationbenchmark BTIO is a scientific workload that is used inthe study of MPI-IO performance. We also used dbenchwhich simulates netbench, an industry standard for testingfile servers.

• Rawio: Rawio is a I/O benchmarking tool that is usedto measure the raw block read and write performanceof disks. The parameters that can be varied includeaccess pattern, number of outstanding requests andthe block size. We choose the combinations of thefollowing parameters: Random/Sequential and blocksize = 512 bytes, 8K , 64K and Reads/Writes. Foreach of the combination, we test the performance ofthe nine design choices by running Rawio concurrentlyfrom 1,2,4 and 8 clients. The maximum number ofoutstanding I/O requests was set to 16 in each of theclients. The measured metric is the average point topoint throughput as seen by a client in MB/sec.

• BTIO: The BTIO benchmark (Version 2.4) from NASAAmes Research Center [5] simulates the I/O requiredby a time-stepping flow solver that periodically writesits solution matrix. The solution matrix is distributedamong processes by using a multi-partition distribu-tion in which each process is responsible for severaldisjoint sub-blocks of points (cells) of the grid. Theaccess pattern in BTIO is non-contiguous in memoryand on file and uses MPI derived data-types to describethis non-contiguity. We ran the Class A problem sizefor our experiments on 1,4 and 9 clients. It uses a64x64x64 element array to generate a file of size 400MB.

• dbench: dbench is an application that simulates net-bench which is an industry standard for testing fileservers. Since the requirements of running netbenchare 60 to 150 Windows PCs on switched fast Ethernet,

dbench is often chosen as an alternative to generate thefile-system load typically seen from a netbench client.The file-system load is created with the same IO callsthat the smbd server in Samba would invoke in a net-bunch run. dbench does not exercise any networkingcalls.

4.4 ResultsFigure 6 shows the performance of Rawio for different

sequential and random workloads on varying block sizes.First, we consider the throughput values obtained for se-quential reads (the leftmost bar stack in the bar charts ofFig 6 (a)) for a single client case with no hardware crypto ac-celeration. When the block size is 512 bytes it was observedthat the maximum throughput of a single running clientwithout any security is around 5.6 MB/sec. It is limitedby the network I/O calls and iSCSI processing overheads ofsmall packets. When IPSec AH (the bar labeled S1-1) andIPSec ESP (S2-1) are running in conjunction with iSCSI,the throughput drops down by 13.4% and 24.8% respec-tively. When the block size is increased, the iSCSI layer andnetwork I/O processing are no longer the bottleneck and themaximum throughput in the No-Security case continues toincrease peaking at the network bandwidth at 64K bytes.However, IPSec processing becomes a bottleneck at largerblock sizes due to the CPU overhead of crypto processingwith throughput reductions of 52.5% and 71% for S1-1 andS2-1.

The bars also show that the lazy approaches and theircombination with IPSec outperform the pure IPSec perfor-mance. Lazy authentication (S1-2) achieves better through-put in the security level S1 with significant improvementsover IPSec AH at larger block sizes (46.2% for 64K bytes).The combination of Lazy decryption and IPSec AH (S2-3)also performs well compared to the pure IPSec ESP case al-though the improvements are not as large as before (17.3%for 64K bytes) The full combination of Lazy decryption andLazy authentication (S2-5) performs even better with im-provements of 11.6% and 40.4% for the small and large blocksizes. The reduction in crypto processing at the server endresults in better throughput for the lazy approaches.

When more than one client are set to request sequentialblocks simultaneously, Figures 6 (a) and (b) show the av-erage point to point throughput seen by a client is reducedby a factor only slightly less than the number of clients.This shows that the disk transfer bandwidth at the serveris not much of a overhead for sequential workloads and thatthe maximum throughput obtained when a single client isrunning scales well when more clients are added. Also notethat the lazy approaches are much more scalable than IPSecat larger block sizes with the increasing clients. For se-quential reads with eight clients running (the rightmost barstack in the bar charts of Figure 6(a)), Lazy authentica-tion achieves a 92.4% improvement over IPSec AH for ablock size of 64K bytes. The corresponding values for thecombinations of Lazy decryption/IPSec AH and Lazy de-cryption/Lazy authentication over IPSec ESP are 44.4% and170%. The tremendous increase in the relative throughputvalues of the lazy approaches compared to IPSec is due tothe fact that there is very little crypto processing overheadat the server. All the data packets are authenticated orencrypted/decrypted at the client side resulting in a veryscalable design.

(a)

(b)

(c)

(d)

Figure 6: Point to Point throughput of Rawio for (a) sequential reads, (b) sequential writes, (c) randomreads, (d) random writes

Table 3: Completion time of BTIO measured in sec-onds

S0 S1-1 S1-2 S2-1 S2-3 S2-5

One client 444.84 446.27 445.36 462.15 454.32 449.13Four clients 144.9 147.28 145.25 152.16 149.51 146.38Nine clients 131.02 133.74 131.96 136.21 134.33 133.6

Table 4: Point-to-point throughput of dbench mea-sured in MB/second

S0 S1-1 S1-2 S2-1 S2-3 S2-5

One client 45.55 39.88 45.35 32.2 36.82 41.74Two clients 27.72 26.75 27.16 21.11 23.33 24.48Four clients 9.45 9.15 9.22 8.23 8.8 9.01Eight clients 3.13 3.04 3.11 2.23 2.75 2.91

For the random workloads (Figures 6(c) and (d)), similarresults can be inferred. The lazy approaches performed bet-ter in all cases compared to the IPSec based ones. For thesingle client case, the relative improvements are less com-pared to the sequential workloads. This is because of thelower bandwidth seen from the disk for random accesses.The maximum throughput for the No-security case is about33 MB/sec for 8K and 52 MB/sec for 64K block sizes anddoes not saturate the Gigabit Ethernet bandwidth. There-fore, apart from the crypto processing, the bottleneck is lim-ited by the disk transfer bandwidth.

The three bars labeled S2-2, S2-4 and S2-6 in the fig-ures are lazy approaches implemented by performing en-cryption/decryption in hardware. It was found that theperformance improvements by using the hardware was notmuch compared to the respective software implementationsS2-1, S2-3 and S2-5. For smaller block sizes, the overheadof asynchronous processing, numerous hardware calls madevia the PCI bus and interrupt processing become a bottle-neck. With larger block sizes, there are minor performancebenefits by off-loading to the crypto hardware but this doesnot scale with increasing number of clients

Table 3 shows the execution time of BTIO for the de-sign spectrum outlined earlier in Table 2. We omit the de-sign choices involving the hardware accelerator because themicro-benchmark results using Rawio indicate that there isnot much benefit in off-loading crypto especially for smallerblock sizes and increasing number of clients (since BTIOwrites its data to the GFS file system which in turn uses asmall block size). As seen from the results, BTIO yields poorperformance benefits with lazy authentication and encryp-tion schemes because of fewer, smaller and well-orchestratedI/O transfers and the reduced proportion of time spent insending and receiving data blocks over the network.

Table 4 shows the throughput for dbench which is a moreI/O bound workload and hence shows better performance re-sults than BTIO. In particular we find that for the Security-1design space, performing lazy authentication (S1-2) booststhroughput to as much as the scheme which does not doany crypto (S0). Likewise for the Security-2 design space,a combination of lazy decryption and authentication (S2-5)boosts throughput to about 90% of the no crypto scheme(S0) and 30% improvement over IPSec ESP.

5. RELATED WORKMuch of the work on storage security has been done at

the file system level. A framework for evaluating security of

diverse systems is discussed in [21], where they classify filesystems into either encrypt-on-wire or encrypt-on-disk sys-tems depending on where the encryption/decryption takesplace. Examples of systems which implement encrypt-on-wire include PASIS [25] and S4 [23], where each transmis-sion requires encryption or decryption. On the other hand,file systems such as CFS [6], SFS-RO [12], Plutus [19], SiR-iUS [15] perform encrypt-on-disk. As pointed out in [21],the latter category provides confidentiality even in an un-trusted server model and can also incur less overheads indata transfer. In our work, the lazy decryption approach isan encrypt-on-disk mechanism with the data stored in anencrypted form on the disks.

At the device level, disk drives have traditionally not pro-vided any security mechanisms. It is usually left to thehigher level software to provision the security mechanisms -either access control or confidentiality/authenticity/integrity.The OSD model [9] with embedded security [17] is one ofthe first proposals to incorporate smartness in the drive forsecurity functionality. A discussion of security considera-tions when deploying Fibre Channel SANs can be foundin [7]. The IP versions of these protocols - iSCSI,iFCP,FCIPare typically expected to use the underlying network layers(IPSec, SSL) for authentication, integrity and confidential-ity. The work by Aguilera et al [4] addresses the accesscontrol mechanisms needed at the block level for networkattached disks. The mechanisms proposed are similar inconcept to the OSD model which provides security at ob-ject granularity.

There has been some work in evaluating the performanceof these secure storage systems that are attached to the net-work. Authors in [17] describe one such system emphasizingon performance and scalability of network attached storagesystems. The file management functions occur at a differentlocation than the storage device. The file managers issuecapabilities to client managers which then access the filesstored on the device directly. A follow up report [16] showshow software-based cryptography cannot support storage’sgigabit/sec transfer rates.

There is a need to secure SAN systems like the iSCSI sys-tems, where the disk is directly attached to the network andblock level accesses are made over the network. The securityof the blocks at rest and in transit must be achieved withminimal performance overheads. [10] performs encryption atthe block level. The proposed framework SNAD relies uponseveral standard public-key cryptographic tools. Howeverthe use of public-key cryptography decreases the throughputappreciably and the evaluation does not include high-levelor standardized benchmarks. [8] performs a detailed studyof iSCSI performance using different micro-benchmarks likeIOmeter and real applications. However the study does notinclude any measurements of security overheads and scala-bility of iSCSI with increase in number of clients. In [22] theauthors analyze the security requirements of iSCSI storagesystems and examine the integration of two popular secu-rity schemes, IPSec and SSL in iSCSI. They also note thatthe two mechanisms show different throughput trends withdifferent data sizes.

Although the performance of existing iSCSI implemen-tations without any security considerations have been wellstudied ([3, 24, 26, 18, 27, 11]), we analyze the performanceof iSCSI with and without security considerations for a setof micro and application benchmarks. To our knowledge,

ours is the first proposal which exploits the secret key ofthe lower level secure protocol to secure communication be-tween machines at the application level without sacrificingperformance.

6. CONCLUDING REMARKSIn this paper, we analyze the security and performance

requirements of iSCSI based network storage systems. Ex-isting techniques like IPSec that provide security at the net-work layer incur large overheads for iSCSI which are basedon the fast growing Ethernet technology. We identify thethe server end crypto processing of data packets as a sig-nificant bottleneck in the overall system throughput. Toalleviate the processing overheads, we propose lazy cryptoapproaches which reduce the amount of work that needs tobe done at the server. Our results show that the lazy tech-niques perform better than the IPSec based ones and resultin better scalability with the increase in number of clients.

7. ACKNOWLEDGEMENTSThis work has been supported by NSF grant 0621429.

This work was partially supported by the Mathematical,Information, and Computational Sciences Division subpro-gram of the Office of Advanced Scientific Computing Re-search, Office of Science, U.S. Department of Energy, underContract W-31-109-Eng-38.

8. REFERENCES[1] Linux iSCSI Target Implementation, ARDIS

Technologies. http://www.ardistech.com/iscsi.

[2] Open-iSCSI Project, A Multi-PlatformImplementation of iSCSI RFC.http://www.open-iscsi.org.

[3] A. Chadda and A. Palekar and R. Russel and N.Ganapathy. Design, implementation and performanceanalysis of the iSCSI protocol for SCSI over TCP/IP.In Proceedings of the Internetworking 2003 conference,2003.

[4] M. Aguilera, M. Ji, M. Lillibridge, J. Maccormick,E. Oertli, D. Andersen, M. Burrows, T. Mann, andC. Thekkath. Block-level Security forNetwork-Attached Disks. In Proceedings of theConference on File and Storage Technologies, 2003.

[5] D. H. Bailey, E. Barszcz, J. T. Barton, D. S.Browning, R. L. Carter, D. Dagum, R. A. Fatoohi,P. O. Frederickson, T. A. Lasinski, R. S. Schreiber,H. D. Simon, V. Venkatakrishnan, and S. K.Weeratunga. The NAS Parallel Benchmarks. TheInternational Journal of Supercomputer Applications,pages 63–73, Fall 1991.

[6] M. Blaze. A Cryptographic File System for UNIX. InProceedings of the ACM Conference on Computer andCommunications Security, pages 9–16, 1993.

[7] E. D. Burgh. SAN Security Beyond Segmentation,2004.http://www.sans.org/rr/whitepapers/honors/1489.php.

[8] D. Xinidis and M. D. Flouris and A. Bilas.Performance Evaluation of Commodity iSCSI-basedStorage Systems. In Proceedings of the Twenty SecondIEEE, Thirteenth NASA Goddard Conference on MassStorage Systems and Technologies, 2005.

[9] T. W. Draft. Information Technology - SCSI ObjectBased Storage Device Commands (OSD).http://www.t10.org/ftp/t10/drafts/osd/osd-r10.pdf.

[10] E. Miller and W. Freeman and D. Long and B. Reed.Strong Security for Network-Attached Storage. InProceedings of the Conference on File and StorageTechnologies, 2002.

[11] F. Tomonori and O. Masanori. Performance ofOptimized Software Implementation of iSCSI. InProceedings of the Workshop on Storage NetworkArchitecture and Parallel I/Os (SNAPI), 2003.

[12] K. Fu, M. F. Kaashoek, and D. Mazieres. Fast andSecure Distributed Read-only File System. InProceedings of the Usenix Symposium on OperatingSystems Design and Implementation, pages 181–196,October 2000.

[13] G. Caronni and H. Lubich and A. Aziz and T.Markson and R. Skrenta. SKIP-securing the Internet.In Proceedings of the 5th International Workshops onEnabling Technologies: Infrastructure for CollaborativeEnterprises (WET ICE’96), page 62, Washington,DC, USA, 1996. IEEE Computer Society.

[14] G. R. Goodson and J. J. Wylie and G. R. Ganger andM. K. Reiter. Efficient Byzantine-TolerantErasure-Coded Storage. In Proceedings of the 2004International Conference on Dependable Systems andNetworks (DSN’04), page 135, Washington, DC, USA,2004. IEEE Computer Society.

[15] E. Goh, H. Shacham, N. Modadugu, and D. Boneh.SiRiUS: Securing Remote Untrusted Storage. InProceedings of the Internet Society Network andDistributed Systems Security Symposium, 2003.

[16] H. Gobioff and D. Nagle and G. Gibson. Integrity andPerformance in Network Attached Storage. TechnicalReport CMU-CS-98-182, Carnegie Mellon University -School of Computer Science, 1998.

[17] H. Gobioff and G. Gibson and D. Tygar. Security forNetwork Attached Storage Devices. Technical ReportCMU-CS-97-185, Carnegie Mellon University - Schoolof Computer Science, 1997.

[18] H. M. Khosravi and A. Joglekar. PerformanceCharacterization of iSCSI processing in a serverplatform. In Proceedings of the Twenty Fourth IEEEInternational Performance Computing andCommunications Conference, 2005.

[19] M. Kallahalla, E. Riedel, R. Swaminathan, Q. Wang,and K. Fu. Plutus – Scalable Secure File Sharing onUntrusted Storage. In Proceedings of the Conferenceon File and Storage Technologies, pages 29–42, March2003.

[20] M. Abd-El-Malek and G. R. Ganger and M. K. Reiterand J. J. Wylie and G. R. Goodson. Lazy Verificationin Fault-Tolerant Distributed Storage Systems. InProceedings of the 24th IEEE Symposium on ReliableDistributed Systems (SRDS’05), pages 179–190,Washington, DC, USA, 2005. IEEE Computer Society.

[21] E. Riedel, M. Kallahalla, and R. Swaminathan. AFramework for Evaluating Storage System Security. InProceedings of the Conference on File and StorageTechnologies, pages 15–30, 2002.

[22] S-Yi Tang and Y. Lu and D. Du. Performance Studyof Software-based iSCSI Security. In Proceedings of

the First International IEEE Security in StorageWorkshop, 2002.

[23] J. D. Strunk, G. R. Goodson, M. L. Scheinholtz,C. A. N. Soules, and G. R. Ganger. Self-SecuringStorage: Protecting Data in Compromised Systems. InProceedings of the Usenix Symposium on OperatingSystems Design and Implementation, pages 165–180,October 2000.

[24] W. T. Ng and H. Sun and B. Hillyer and E. Shiver andE. Gabber and B. Ozden. Obtaining high performancefor storage outsourcing. In Proceedings of the 2001ACM SIGMETRICS Conference on Measurement andModeling of Computer Systems, 2001.

[25] J. J. Wylie, M. W. Bigrigg, J. D. Strunk, G. R.Ganger, H. Kilite, and P. K. Khosla. SurvivableInformation Storage Systems. IEEE Computer,33(8):61–68, August 2000.

[26] X. He and Q. Yang. A Caching Strategy to improveiSCSI performance. In Proceedings of the TwentySeventh Annual IEEE International Conference onLocal Computer Networks, 2002.

[27] Y. Shastry and S. Klotz and R. Russell. Evaluatingthe effect of iSCSI protocol parameters onperformance. In Proceedings of the Parallel andDistributed Computing and Networks, 2005.