sqrrl september webinar: cell-level security

sqrrl Secure.'Scale.'Adapt.'

Sqrrl Data, Inc. All Rights Reserved

Sqrrl,&Apache&Accumulo,&and&Cell3Level&Security&

Adam'Fuchs,'CTO'Sqrrl'Data,'Inc.'

September'12,'2013'

2'Sqrrl Data, Inc. All Rights Reserved

•  Introduc@on'to'Sqrrl'and'Accumulo'

•  Security'In'The'Wild'

•  Sqrrl'and'Accumulo'Technology'

•  The'DataECentric'Security'Ecosystem'

•  How'to'Learn'More'

Outline

3'Sqrrl Data, Inc. All Rights Reserved

•  What'is'your'level'of'familiarity'with'Accumulo?'a.  Only'heard'of'it'b.  Well'read'on'it'

c.  Downloaded'and'experimented'with'it'

d.  Currently'using'it'

Poll

4'Sqrrl Data, Inc. All Rights Reserved

Who We Are

Google’s(BigTable(Paper(

2006&

NSA(Builds(Accumulo(

2008&

NSA(Open(Sources(Accumulo(

2011&

Sqrrl(Founded(2012&

3rd(Sqrrl(Release(September&2013&

Investors

5'Sqrrl Data, Inc. All Rights Reserved

Security'

Adap@vity'Scalability'

The Value of Sqrrl and Accumulo

6'Sqrrl Data, Inc. All Rights Reserved

•  Introduc@on'to'Sqrrl'and'Accumulo'

•  Security'In'The'Wild'

•  Sqrrl'and'Accumulo'Technology'

•  The'DataECentric'Security'Ecosystem'

•  How'to'Learn'More'

Outline

7'Sqrrl Data, Inc. All Rights Reserved

Secure Data Lake Concept

NonEsensi@ve'data'Sensi@ve'data'Highly'sensi@ve'data'Highly'sensi@ve'data'Sensi@ve'data'NonEsensi@ve'data'NonEsensi@ve'data'

Real>Time(Apps(

Sqrrl(Enterprise(

8'Sqrrl Data, Inc. All Rights Reserved

Healthcare Security Requirements

PII(

SensiBve(Diagnoses(

Doctor’s(Notes(

9'Sqrrl Data, Inc. All Rights Reserved

•  Introduc@on'to'Sqrrl'and'Accumulo'

•  Security'In'The'Wild'

•  Sqrrl'and'Accumulo'Technology'

•  The'DataECentric'Security'Ecosystem'

•  How'to'Learn'More'

Outline

10'Sqrrl Data, Inc. All Rights Reserved

Sqrrl Enterprise Built on Apache Accumulo

Sqrrl(Server(

Bulk%Processing%Integra2on%

Exploratory%/%Opera2onal%Apps%

Graph%+%Document%I/O%

Sqrrl&API&over&Apache&ThriC&RPC&(JSON,(Graph,(AggregaBon,(Search,(etc.)(

•  Sqrrl(proprietary(•  Automated(indexing(•  Custom(iterators(•  Lucene(integraBon(•  Security(extensions( Accumulo&RPC&

(Sorted(Key/Value(I/O)(

Hadoop&RPC&(File(I/O)(

•  Open(source((including(Sqrrl(contribuBons)(

•  Open(source(or(commercial(distribuBons(

11'Sqrrl Data, Inc. All Rights Reserved

Hadoop(Distributed(File(System((commercial(or(open(source)(

Commodity(Hardware(( Private(Cloud( Public(Cloud(

Sqrrl(AnalyBcs(Sqrrl(Security(

Apache(Accumulo(

Data(Structures(

Languages(

Processing(

Sqrrl(Data(Loaders( Lucene(

Documents((JSON)(

EncrypBon>At>Rest(

EncrypBon>In>MoBon(

Audit(

IdM(IntegraBon( Indexing(Tools(

Policy(&(Labeling(Engines(

Sqrrl&Enterprise&

SQL(Subset(

MapReduce(Connector(

Pig(Connector(

(Sqrrl(Iterators(

User(Interface(D3(Demos(

Flume(

ThriY(API(

Sqrrl(Ingest(

Graphs(

Sqrrl Architecture

12'Sqrrl Data, Inc. All Rights Reserved

An(Accumulo(key(is(a(5>tuple,(consisBng(of:((

"   Row:(Controls'Atomicity'"   Column(Family:(Controls'Locality''"   Column(Qualifier:((Controls'Uniqueness'"   Visibility(Label:((Controls'Access'"   Timestamp:((Controls'Versioning'

Row( Col.(Fam.( Col.(Qual.( Visibility( Timestamp( Value(

John'Doe' Notes' PCP' PCP_JD' 20120912'Pa@ent'suffers'from'an'acute'…'

John'Doe' Test'Results' Cholesterol' JD|PCP_JD' 20120912' 183'

John'Doe' Test'Results' Mental'Health' JD|PSYCH_JD' 20120801' Pass'

John'Doe' Test'Results' XERay' JD|PHYS_JD' 20120513' 1010110110100…'

Accumulo(Key/Value(Example(

Accumulo Data Format

13'Sqrrl Data, Inc. All Rights Reserved

Accumulo Technology

InEMemory'Map'

Write'Ahead'Log'

(For'Recovery)'

Sorted,'Indexed'File'

Sorted,'Indexed'File'

Sorted,'Indexed'File'

Tablet(Data(Flow(

Reads%Iterator'Tree'

Minor%Compac2

on%

Merging%/%Major%Compac2on%

Iterator'Tree'

Writes% Iterator'Tree'

Scan%

Fate'States'

Tablet'Server'

Tablet'

Tablet'Server'

Tablet'

Tablet'Server'

Tablet'

Applica@on'

Zookeeper'

Zookeeper'

Zookeeper'

Master'

HDFS'

Read/Write%

Store/Replicate%

Assign/Balance%

Delegate%Authority%

Delegate%Authority%

Applica@on'

Applica@on'

14'Sqrrl Data, Inc. All Rights Reserved

Table Design Patterns

Table:(

Row:(

Column(Family:(

Column(Qualifier:(

Value:(

Forward(Index(

<UUID>(

<Type>(

<Field>(

<Term>(

Inverted(Index(

<Term>(

<Type>(+(<Field>(

<UUID>(

<Digest(of(Event>(

Table:(

Row:(

Column(Family:(

Column(Qualifier((Tuples):(

Value:(

Shard(Table(

<ParBBon(ID>(

“Docs”( “Inv.(Index”( “Field(Index”(

<UUID>(

<Value>(

<Term>(

<UUID>(

<Field:Term>(

<UUID>(<Field>(

“Geo”(

<Hash>(

<UUID>(

Event( Term(

Ingest(Process( Query(Process(

Indexed(Event(Table(

Event(Columns(

Index(Columns(

Event(Columns(

Index(Columns(

Event(Columns(

Index(Columns(

Event(Columns(

Index(Columns(

Event(Columns(

Index(Columns(

15'Sqrrl Data, Inc. All Rights Reserved

•  Introduc@on'to'Sqrrl'and'Accumulo'

•  Security'In'The'Wild'

•  Sqrrl'and'Accumulo'Technology'

•  The'DataECentric'Security'Ecosystem'

•  How'to'Learn'More'

Outline

16'Sqrrl Data, Inc. All Rights Reserved

Data-Centric Security

Row Col Value 1 Name Jones

1 Sales 100

1 Age 28

2 Name Smith

2 Sales 350

2 Age 25

2' Quota' 1000'

Row Col Value 1 Name Anon1

1 Sales 100

2 Name Smith

2 Sales 350

2' Age' 25'

2' Quota' 1000'

User 1 User 2 Sqrrl/(

Accumulo(

DefiniBon:'Data'carries'with'it'informa@on'that'is'required'to'make'policy'decisions'on'its'releasability.'

17'Sqrrl Data, Inc. All Rights Reserved

Security for Transformed Data

Logs/Observa@ons'

Input'

Indexes'

Ques@onEFocused'Datasets'

Transforma@

on'

Simple(Provenance:(Row(+(Column(Security(OK(

Complex(Provenance:(Cell>Level(Security(Needed(

18'Sqrrl Data, Inc. All Rights Reserved

Security

Row( Col.(Fam.( Col.(Qual.( Visibility( Timestamp( Value(

John'Doe' Notes' PCP' PCP_JD' 20120912'Pa@ent'suffers'from'an'acute'…'

John'Doe' Test'Results' Cholesterol' JD|PCP_JD' 20120912' 183'

John'Doe' Test'Results' Mental'Health' JD|PSYCH_JD' 20120801' Pass'

John'Doe' Test'Results' XERay' JD|PHYS_JD' 20120513' 1010110110100…'

Example(Accumulo(Key/Value(Pairs(

Accumulo&is&the&only&NoSQL&database&with&cell3level&access&controls&

19'Sqrrl Data, Inc. All Rights Reserved

•  JSON'maps'to'document'and'index'entries,'all'of'which'preserve'the'security'labels'

•  Labels'follow'the'document'hierarchy'•  The'label'is'part'of'the'name'of'the'field'(affects'uniqueness)'

JSON Document Security Labels

20'Sqrrl Data, Inc. All Rights Reserved

Data-Centric Security Ecosystem

Data( Labeler( Sqrrl(Enterprise(

Apps(

User(Acributes(

Audits(

Policies(

End(Users(

Auth.(Service(

Policy(Engine(

Key(Mgmt(

21'Sqrrl Data, Inc. All Rights Reserved

•  Introduc@on'to'Sqrrl'and'Accumulo'

•  Security'In'The'Wild'

•  Sqrrl'and'Accumulo'Technology'

•  The'DataECentric'Security'Ecosystem'

•  How'to'Learn'More'

Outline

22'Sqrrl Data, Inc. All Rights Reserved

hip://accumulo.apache.org'

Current'Developer'Base:'•  18'Commiiers'•  39'Addi@onal'Recognized'

Contributors'Contribu@ng'Organiza@ons:'

'Sqrrl,'Koverse,'Basis,'Hortonworks,'Praxis,'NSA,'Texeltek,'Objec@ve'Solu@ons,'Booz'Allen'Hamilton,'SW'Complete,'Endgame,'SRA,'Peterson'Technologies,'Cloudera,'Agile'Technology'Group,'Data'Tac@cs,'Tetra'Concepts,'JHU/APL,'Applied'Technical'Systems,'and'more.'

Accumulo Community Growth

23'Sqrrl Data, Inc. All Rights Reserved

•  Download'our'White'Paper'– www.sqrrl.com/whitepaper'

•  Watch'a'video'– www.sqrrl.com/downloads#videos'

•  Request'a'demo'or'oneEonEone'workshop'–  'www.sqrrl.com/contact'

•  Come'meet'us'– Strata'Rx'(Sept'25E27,'Boston)'– Hadoop'World'(Oct'28'–'30,'New'York)'

How To Learn More

24'Sqrrl Data, Inc. All Rights Reserved

Thank you

Thanks(for(acending!(

To(keep(up(to(date(with(Sqrrl,(check(out(or(social(media(sites:(www.twicer.com/sqrrl_inc(

www.linkedin.com/company/sqrrl((