security with elasticsearch - goto...
TRANSCRIPT
Agenda
Why?How?
Next?
What?
Who?
Q & A
About
2012
Elasticsearch got founded
Series A investment
Trainings
Supports subscriptions
About
2012
Series B investment
Kibana
Elasticsearch for Apache Hadoop Integration
Logstash
Elasticsearch Clients
2013
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant logo are trademarks of the Apache Software Foundation in the United States and/or other countries.
About
2012
Series C investment
Marvel released
2013 2014
About
2012
Shield goes GA
First user conference & rebrand
Found acquired
Packetbeat joins
Watcher in beta
2013 2014 2015
About
2012 2013 2014 2015
Joined in March 2013
Working on Elasticsearch & Shield
Development, Trainings, Conferences, Support, Blog posts
We're hiring...
Why?How?
Next?
What?
Who?
Q & A
Why?
Elasticsearch: No security OOTB
No encrypted communication
No Authorization
No Authentication
No Audit Logging
ESnginxclient
Filter by HTTP method, URI or IP
User management via basic auth
Use aliases & filters
nginx in front
ESnginxclient
How to solve multi index operations?
nginx in front
GET /logs-2015.10.10,evil,logs-2015.10.11{ "query" : { "match_all": {} }}
ESnginxclient
How to solve bulk/multi operations?
nginx in front
{ "index" : { "_index" : "test1", "_type" : "type1", "_id" : "1" } }{ "field1" : "value1" }{ "delete" : { "_index" : "test2", "_type" : "type1", "_id" : "2" } }{ "create" : { "_index" : "test3", "_type" : "type1", "_id" : "3" } }{ "field1" : "value3" }{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} }{ "doc" : {"field2" : "value2"} }
ESnginxclient
Prevent unwanted accesses
nginx in front
HTTP/Transport
ESnginxclient
nginx in front
Firewall
DataACLclient
Configuration scattered across systems
operational overhead
IP Filtering
DataACLclient
operational overhead
IP Filtering Directory
Configuration scattered across systems
Why?How?
Next?
What?
Who?
Q & A
How?
Elasticsearch modular & pluggable
Security as a plugin
HTTP + Transport protocols
Integration into the ELK stack!
How?
ElasticsearchElasticsearch
auth_token Authentication
Authorization
How?
ElasticsearchElasticsearch
auth_token
200 OK
Authentication
Authorization
How?
ElasticsearchElasticsearch
auth_token
401 Unauthorized
Authentication
Authorization
How?
Getting up and running is easy
Install elasticsearch 1.6
bin/plugin install elasticsearch/license/latest
bin/plugin install elasticsearch/shield/latest
Why?How?
Next?
What?
Who?
Q & A
What?
IP Filtering
Encrypted communication
Authentication
Authorization
Audit Trail
Configurable in elasticsearch.yml
Can be updated dynamically via cluster update settings API
IP Filtering
shield.transport.filter: allow: "192.168.0.1" deny: "192.168.0.0/24"
keystore required
different config for HTTP and transport protocol (+profiles)
Encrypted communication
shield.ssl.keystore.path: /path/to/keystore.jksshield.ssl.keystore.password: secretshield.transport.ssl: trueshield.http.ssl: true
Authentication
"Who are you?"
Auth mechanisms are called realms
Available: esusers, ldap, ad, pki
Realms can be chained
Support for caching & API for clearing
Authenticationshield.authc: realms: esusers: type: esusers order: 0
ldap1: type: ldap order: 1 enabled: false url: 'url_to_ldap1' ...
ad1: type: active_directory order: 3 url: 'url_to_ad'
ESusers realm
Local files, can be changed via CLI
Elasticsearch watches file changes & reloads
config/shield/users
config/shield/users_roles
ESusers realm
bin/shield/esusers useradd alex
bin/shield/esusers roles alex -a admin -r user
bin/shield/esusers list
bin/shield/esusers userdel alex
Fallback to configurable user
Disabled by default
Anonymous access
shield.authc: anonymous: username: anonymous_user roles: role1, role2
Authorization
"Are you allowed to do that?"
File: config/shield/roles.yml
admin: cluster: all indices: '*': all
Role Based Access Control
role named set of permissions
permission set of cluster wide privileges
set of indices/aliases specific privileges
privilege set of one or more action names
/_search ⬌ indices:data/read/search
Role Based Access Control
admin: cluster: all indices: '*': all
role permission
Authorization
user: indices: '*': read
events_user: indices: 'events_*': read
Authorization
get_user: indices: 'events_index': 'indices:data/read/get'
logfile_user_readonly: indices: "logstash-201?-*": read
Audit Trail
Writes an own audit log file
Implemented as logger
Logs different types of event based on log level (ip filtering, tampered requests, access denied, auth failed)
shield.audit.enabled: true
Integration
Transport Client
Logstash
Kibana 3/4
Watcher
Marvel
Transport Client
TransportClient client = new TransportClient(builder() .put("cluster.name", "myClusterName")
.put("shield.user", "test_user:changeme")
.put("shield.ssl.keystore.path", "/path/to/client.jks") .put("shield.ssl.keystore.password", "password") .put("shield.transport.ssl", "true"))
.addTransportAddress(new InetSocketTransportAddress("localhost", 9300));
Why?How?
Next?
What?
Who?
Q & A
Who?
Use-case 1: Monitoring application
No write access
Cluster Health Nodes stats/info Indices Stats
Use-case 2: Logstash
No read access (unless input is used)
Indices: Indexing
Cluster: Index templates
Use-case 3: Marvel
marvel_user: cluster: cluster:monitor/nodes/info, cluster:admin/plugin/license/get indices: '.marvel-*': all
marvel_agent: cluster: indices:admin/template/get, indices:admin/template/put indices: '.marvel-*': indices:data/write/bulk, create_index
Use-case 4: Ecommerce
bulk: indices: 'products_*': write, manage, read
updater: indices: 'products': index, delete, indices:admin/optimize
webshop: indices: 'products': search, get
Use-case 4: Ecommerce
monitoring: cluster: monitor indices: '*': monitor
sales_rep : indices: 'sales_*' : all 'social_events' : data_access, monitor
Why?How?
Next?
What?
Who?
Q & A
Next?
Simplify SSL configuration
API driven user/role management
Open up realms API
Field-level security
Index Audit Trail into ES
Why?How?
Next?
What?
Who?
Q & A
Q & A
Thanks for listening!
Alexander Reelsen @spinscale
We're hiring https://www.elastic.co/about/careers
We're helping https://www.elastic.co/subscriptions
Resources
Shield documentation https://www.elastic.co/guide/en/shield/current/index.html
Shield: Security in ELK https://www.elastic.co/elasticon/2015/sf/security-in-elk
Shield and Beyond: Recommendations for a Secure ELK Environment https://www.elastic.co/webinars/shield-and-beyond
Resources
Resources
Q & A
Thanks for listening!
Alexander Reelsen @spinscale
We're hiring https://www.elastic.co/about/careers
We're helping https://www.elastic.co/subscriptions