building an encrypted and searchable audit log
DESCRIPTION
Building an Encrypted and Searchable Audit Log. Brent Waters Dirk Balfanz Glenn Durfee D.K. Smetters. Audit Logs. Employed on most server systems Web logs Database logs Provide invaluable access to past activity Hold users accountable for their actions Diagnostics . - PowerPoint PPT PresentationTRANSCRIPT
Building an Encrypted and Searchable Audit Log
Brent Waters
Dirk BalfanzGlenn DurfeeD.K. Smetters
Audit Logs
• Employed on most server systems– Web logs– Database logs
• Provide invaluable access to past activity– Hold users accountable for their actions– Diagnostics
Desirable Characteristics
• Tamper Resistant• Verifiable
– Can check that entries are present and have not been altered
• Data Access Control– Entries may be sensitive to individuals or log owner
• Searchability– Search for log on specific criteria– e.g keyword search
Desirable Characteristics
• Tamper Resistant• Verifiable
– Can check that entries are present and have not been altered
• Data Access Control– Entries may be sensitive to individuals or log owner
• Searchability– Search for log on specific criteria– e.g keyword search
An Audit Log for a Database System
user: Alice Smithkeyword: carskeyword: makekeyword: fordtime: 2003/08/26 23:34:24
authentication clockkeyword extraction
“select * from cars where make=‘ford’”
databaseaudit record creation
keywords for audit record
log storage (untrusted)
Requirements
• Data Access Control– Entries must be encrypted on untrusted storage– Forward security in case auditing device becomes
compromised asymmetric encryption– Limit scope of data released to that of the search
• Searchability– Be able to efficiently retrieve entries based on certain criteria– We focus on keyword search
A Simple Solution
• Encrypt all entries with a public key• Auditor downloads all entries, then decrypts them,
then performs the search
A Simple Solution
• Encrypt all entries with a public key• Auditor downloads all entries, then decrypts them,
then performs the search
Disadvantages• Auditor sees all entries and regardless of what
search criteria was• All entries must be transmitted from server
auditrecord
Delegating Search Capabilities
investigator audit escrow agent
mastersecret
“user: Alice Smith”
capabilityfor search
investigator audit log
capabilityfor search
auditrecord
auditrecord …
1
2
The investigator submits the capability to the audit log and receives only entries that the capability matches.
The investigator requests a capability to search for all entries that were made by the user Alice.
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Encrypted Data
Keywords must not be in the clear!
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
audit escrow agent
mastersecret
Encrypted Data
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Honda
Search Capability
mastersecret
Encrypted Data
audit escrow agent
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Honda
Search Capability
mastersecret
Encrypted Data
audit escrow agent
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Honda
Search Capability
mastersecret
Encrypted Data
No information is learned
audit escrow agent
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
mastersecret
Encrypted Data
audit escrow agent
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Alice
Search Capability
mastersecret
Encrypted Data
audit escrow agent
Searching on Asymmetrically Encrypted Data
Auditing Device
Keywords
Alice
Ford
Loans
Document
Alice
Search Capability
mastersecret
Encrypted DataKeywords
Alice
Ford
Loans
Document
Embed decryption in search
audit escrow agent
Identity Based Encryption (IBE)
• Public Key is simply a string e.g. [email protected]
• Private Key given from master secret holder(s)
• Removes need for distribution of public key certificates
• We use scheme of Boneh and Franklin (2001)
Using IBE to Search on Asymmetrically Encrypted Data
Keywords
Alice
Ford
Loans
Document Auditing Device
Using IBE to Search on Asymmetrically Encrypted Data
Auditing DeviceKeywords
Alice
Ford
Loans
Document
Document
K
Using IBE to Search on Asymmetrically Encrypted Data
Auditing DeviceKeywords
Alice
Ford
Loans
Document
Document
K
FLAG | K“Alice”
Using IBE to Search on Asymmetrically Encrypted Data
Auditing DeviceKeywords
Alice
Ford
Loans
Document
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
Using IBE to Search on Asymmetrically Encrypted Data
Auditing DeviceKeywords
Alice
Ford
Loans
Document
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Auditing Device
•FLAG used to test
K to decrypt on match
Keywords
Alice
Ford
Loans
Document
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Auditing Device
•FLAG used to test
K to decrypt on match
•Key-privacy propertykeywords kept private
Keywords
Alice
Ford
Loans
Document
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Auditing Device
•FLAG used to test
K to decrypt on match
•Key-privacy propertykeywords kept private
•“Pairing” operation per keyword
Keywords
Alice
Ford
Loans
Document
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAGDocument
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAG
011010…
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAG
0011100…
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAG
FLAG | K
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAG
•On match use K to decrypt document
DocumentDocument
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
FLAG | K
Using IBE to Search on Asymmetrically Encrypted Data
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAG
•On match use K to decrypt document
•Pairing per keyword in document
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Document
FLAG | K
Scoping of Keywords
• We want to type keywords
• e.g. Capability to search on entries about “Alice” vs. those made by “Alice”
• Solution: Prefix keywords with type– “user:Alice”– “kw:Alice”
Performance
• Encryption– One pairing per keyword in document– One exponentiation per keyword
• Search/Decryption– One pairing per keyword per document
Optimizations
• Cache pairings of frequently used keywords– eg. ê(“user:Alice”,sP)– Only need a pairing per new keyword on encryption– In limit exponentiation per keyword is dominant cost
Optimizations
• Cache pairings of frequently used keywords– eg. ê(“user:Alice”,sP)– Only need a pairing per new keyword on encryption– In limit exponentiation per keyword is dominant cost
• Reuse randomness for IBE encryption within one document– Okay since cannot use same public key per document– In decryption only one pairing per document– Save storage in log
Indexing
• Incremental update of an index on untrusted storage is insecure
Indexing
• Incremental update of an index on untrusted storage is insecure
Keywords
Alice
Ford
Loans
Document Index
Indexing
• Incremental update of an index on untrusted storage is insecure
Keywords
Alice
Ford
Loans
Document Index
Indexing
• Incremental update of an index on untrusted storage is insecure
Keywords
Alice
Loans
Washington
Document Index
Indexing
• Incremental update of an index on untrusted storage is insecure
Keywords
Alice
Loans
Washington
Document Index
Indexing• Build local index on auditing device and flush out
to storage
Indexing• Build local index on auditing device and flush out
to storage
Document
K’
Document
K’’
Document
K
FLAG | K | K’’ “Alice”
FLAG | …“Sam”
Indexing• Longer index is held in auditing device more
information leaked on device compromise
Implementation
• Implemented a logging system for MySQL database queries
• Goal to protect individual’s privacy
• Used Stanford IBE library
• Pairing cost ~80ms on current machines
Related Work
Searching on Encrypted Data• Boneh, Crescenzo, Ostrovsky and Persiano (2003)• Song, Wagner and Perrig (2000)• Goh (2003)
Identity Based Encryption• Boneh and Franklin (2001)
Conclusion
• Tension between data access control and searchability in audit logs
• Asymmetric scheme for searching on encrypted data
• Explored optimizations for practical systems
Searching on Asymmetrically Encrypted Data
Auditing Device
KeywordsAlice
Ford
Loans
Document
KeywordsAlice
Ford
Loans
Document
Bob
Search Capability
audit escrow agent
mastersecret
Encrypted Data
Using IBE to Search on Asymmetrically Encrypted Data
KeywordsAlice
Ford
Loans
Document Auditing Device
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
•FLAG used to test
K to decrypt on match
•Key-privacy propertykeywords kept private
•“Pairing” operation per keyword
Using IBE to Search on Asymmetrically Encrypted Data
Document
K
FLAG | K“Alice”
FLAG | K“Ford”
FLAG | K“Loans”
Alice
Search Capability
•Attempt IBE decryption on each part
Test for presence of FLAG
•On match use K to decrypt document
•Pairing per test