securing cassandra for compliance
TRANSCRIPT
Securing Cassandrafor Compliance (or Paranoia)
Hi, I'm Nate. @zznate https://www.linkedin.com/in/zznate http://www.slideshare.net/zznate/
Co-Founder, CTO The Last Pickle
Cassandra user since 2009 (v0.4) Austin, Texas
Security presentations can be scary. Here's a cat.
First, how did we get here and why is securing Cassandra important?
"Target CEO Gregg Steinhafel Resigns In Data Breach Fallout"
http://www.forbes.com/sites/clareoconnor/2014/05/05/target-ceo-gregg-steinhafel-resigns-in-wake-of-data-breach-fallout/
First, how did we get here and why is securing Cassandra important?
I have
your personal
information
Customers place a lot of trust in technology companies
LOL! Me too!
Sometimes too much.
Ease of scalability comes with a price
HA! A bin-packed message format with no source
verification!*
Ease of scalability comes with a price
* <currently reading o.a.c.net.MessageIn#read>
nmap -Pn -p7000 \-oG logs/cass.gnmap 54.88.0.0/14
I'm publicly discussing your
technical shortcomings
Then you end up in this situation.
Meanwhile, at the FCC...
We have to require two factor, secure socket transport
encryption, something something...
ZZZzzzzzzzZZZzz
We did a regulation!
My staffers still print out my email :)
Why are we doing this again?
Sssshhhh. I'm AES'ing...
...even though the traffic never leaves a backplane.
Some industries will require node to node SSL
1. Encrypting data at rest2. Encrypting data on the wire3. Authentication and authorization4. Management and tooling
Focusing our Discussion: Architecture
1. Encryption at rest
No matter what: understand the failure modes
bit rot, entropy, etc. Horrible things can happen with on disk encryption.
Don't mind me, I'm just your key server.
Haha! Later!
xWhat's on this
disk again?
Shrug.
...but you may not have a choice.
Because we said "at rest"
dmcrypt, eCryptFS
Open source options:
Vormetric, Gazzang
Commercial options:
DSE Encryption
CREATE TABLE users ...WITH compression_parameters:sstable_compression = 'Encryptor' and compression_parameters:cipher_algorithm = 'AES/ECB/PKCS5Padding' and compression_parameters:secret_key_strength = 128;
DSE Encryption
CREATE TABLE users ...WITH compression_parameters:sstable_compression = 'Encryptor' and compression_parameters:cipher_algorithm = 'AES/ECB/PKCS5Padding' and compression_parameters:secret_key_strength = 128;
WARNING:
commitlog not included*
*eCryptFS would work fine for this
EBS Encryption (a.k.a "not my problem")
(Looks like this)
EBS Encryption (a.k.a "not my problem")
http://www.slideshare.net/AmazonWebServices/bdt323-amazon-ebs-cassandra-1-million-writes-per-second
See Crowdstrike's presentation on Cassandra GP2 performance (with encryption):
Maybe Client Side?
The Java Driver now has custom codecs which would make this easy to implement
https://github.com/datastax/java-driver/tree/3.0/manual/custom_codecs
Maybe Client Side?
The Java Driver now has custom codecs which would make this easy to implement
https://github.com/datastax/java-driver/tree/3.0/manual/custom_codecs
Column-level encryption!
New in Cassandra 3.4 (DSE 5.1?):
Commitlog Encryption: CASSANDRA-6018 Hint File Encryption: CASSANDRA-11040
https://issues.apache.org/jira/browse/CASSANDRA-6018https://issues.apache.org/jira/browse/CASSANDRA-11040
2. Encryption on the wire
Because:It is really easy to attack an un-protected cluster
It takes a single Message to insert an admin account
into the system table
-Dcassandra.write_survey=true
How to steal writes in real time:
The fix is straight forward:
node to node encryption and SSL client certificate authentication to cluster traffic
Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate authentication to cluster traffic
Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate authentication to cluster traffic
Bonus: can be done
with NO downtime!!!
Awwwwww.
The fix is straight forward:
node to node encryption and SSL client certificate authentication to cluster traffic
Bonus: can be done
with NO downtime!!!
How-to guide: http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-
server.html
When you are done it should look like:
Things to note:
Use "dc" or "rack" to limit encryption to connections between racks and data centers
Thanks for that!!
Huzzah!
(But AES on modern hardware will not be a bottleneck)
Things to note:
Keystore and key password must match (artifact of JDK X.509 Impl complexity)
Things to note:
256 bit means export restrictions (requires JCE provider JAR)
http://www.oracle.com/technetwork/java/javase/downloads/jce8-download-2133166.html
http://docs.oracle.com/javase/8/docs/technotes/guides/security/SunProviders.html#importlimits
Don't forget this part or else...
Things to note:
Hahaha! Now I'm hacking you over SSL.
*Still* vulnerable AND you can't see what the attacker is doing.
Client to Server SSL
Client to Server SSL(see slides 30 to 35)
Client to Server SSL(see slides 30 to 35)
Now with NO downtime!!!
https://issues.apache.org/jira/browse/CASSANDRA-10559Available in: 2.1.12, 2.2.4, 3.0.0
Need to Debug SSL?
-Djavax.net.debug=ssl
http://docs.oracle.com/javase/7/docs/technotes/guides/security/jsse/ReadDebug.html
Certs are hard :(
Netflix Lemur:x.509 Certificate Orchestration Framework
http://techblog.netflix.com/2015/09/introducing-lemur.htmlhttps://github.com/Netflix/lemur
Certs are hard :(
Hashicorp Vault"secures, stores, and tightly controls access to tokens, passwords, certificates, API keys, and other secrets in modern computing. "
https://www.vaultproject.io/
2. Encryption on the wire
But wait! There's more!
The internode authentication API:BYO identity verification
Looks like this:
3. Authentication and Authorization
Best practices should not be new to you.
user segmentation schema access limitation etc.
(Everything we did with an RDBMS)
Best practices should not be new to you.
user segmentation schema access limitation etc.
Best practices should not be new to you.
user segmentation schema access limitation etc.
(Everything we did with an RDBMS)
New in 2.2:
Role-based access control!
An Example
An Example
An Example
An Example
An Example
buzzword compliant!
An Example
An Example
Turning it all on
authenticator: PasswordAuthenticator
Tip: keep your read-only cqlsh credentials in $HOME/.cassandra/cqlshrc
of the system's admin account
Turning it all on
authorizer: CassandraAuthorizer
Turning it all on
role_manager: CassandraRoleManager
Turning it all on
authorizer: CassandraAuthorizerauthenticator: PasswordAuthenticatorrole_manager: CassandraRoleManager
WARNING:
potential downtime!
authorizer: CassandraAuthorizerauthenticator: PasswordAuthenticatorrole_manager: CassandraRoleManager
Turning it all on
WARNING:
potential downtime!WARNING:
stupid defaults
authorizer: CassandraAuthorizerauthenticator: PasswordAuthenticatorrole_manager: CassandraRoleManager
Turning it all on
WARNING:
potential downtime!WARNING:
stupid defaults
TIP: turn these WAY UP: permissions_validity_in_ms roles_validity_in_ms
Also: use permissions_update_interval_in_ms for async refresh if needed
authorizer: CassandraAuthorizerauthenticator: PasswordAuthenticatorrole_manager: CassandraRoleManager
Turning it all on
WARNING:
potential downtime!WARNING:
stupid defaults
NEW in 3.4:credentials_validity_in_ms*
* https://issues.apache.org/jira/browse/CASSANDRA-7715
Turning it all on
authorizer: TransitionalAuthorizerauthenticator: TransitionalAuthenticator
DSE plugins to avoid downtime
Turning it all on
system.schema_keyspace system.schema_columns system.schema_columnfamilies system.local system.peers
These tables have default read permissions for every authenticated user:
Turning it all on
IMPORTANT cassandra.yaml line note:
"Please increase system_auth keyspace replication factor if you use this..."
Tip: replication factor for the system_auth keyspace should be the same as the number
of nodes in the data center
Turning it all on
IMPORTANT cassandra.yaml line note:
"Please increase system_auth keyspace replication factor if you use this..."
Tip: replication factor for the system_auth keyspace should be the same as the number
of nodes in the data center
WARNING:
stupid defaults*
*https://issues.apache.org/jira/browse/CASSANDRA-11340
4. Management and tooling
4. Management and tooling
Securing JMX
nmap -Pn -p7199 \-oG logs/cass.gnmap 54.88.0.0/14
Always a few suckers that TL,DR'ed
Why do I need to secure JMX?
Works as Advertised!
also good for
some LOLs
Securing JMX
SSL setup is like node to node and client to server
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
Securing JMX
JMX Authentication is straightforward and well documented
$JAVA_HOME/jre/lib/management/jmxremote.access$JAVA_HOME/jre/lib/management/jmxremote.password.template
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access$JAVA_HOME/jre/lib/management/jmxremote.password.template
Now you can:
nodetool -u admin -pw secret compactionstats
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
JMX Authentication is straightforward and well documented
Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access$JAVA_HOME/jre/lib/management/jmxremote.password.template
Now you can:
nodetool -u admin -pw secret compactionstats
Tip: -pwf option will read the password from a file
http://docs.oracle.com/javase/8/docs/technotes/guides/management/agent.html
JMX Authentication is straightforward and well documented
Securing JMX
$JAVA_HOME/jre/lib/management/jmxremote.access$JAVA_HOME/jre/lib/management/jmxremote.password.template
Now you can:
nodetool -u admin -pw secret compactionstats
JMX Authentication is straightforward and well documented
THIS JUST IN!!!
RBAC for JMX Authentication and Authorization
https://issues.apache.org/jira/browse/CASSANDRA-10091
Thanks!@zznate