accumulo summit 2014: accumulo visibility labels and pluggable authorization systems: a love story

60
Securely explore your data Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story John Vines Engineer Sqrrl Data, Inc. [email protected]

Upload: accumulo-summit

Post on 01-Nov-2014

381 views

Category:

Technology


2 download

DESCRIPTION

Labels in Accumulo provide great power and flexibility. However, nearly everyone makes the same set of mistakes when first applying labels to their data. In this talk, we will follow two data architects as they first come to the labeling system in Accumulo, and see how they work their way out of the pitfalls they create for themselves. Along the way, they'll learn about Accumulo's pluggable security architecture surrounding the core functionality of the labeling system.

TRANSCRIPT

Page 1: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

Securely explore your data

Accumulo Visibility Labels and

Pluggable Authorization Systems:A Love Story

John VinesEngineerSqrrl Data, [email protected]

Page 2: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

WHAT MAKES ACCUMULO SPECIAL WHEN IT COMES TO SECURITY?

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Page 3: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

CELL-LEVEL SECURITY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Page 4: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

CELL-LEVEL SECURITY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Page 5: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

tldr;

visibilities are like ACLs

CELL-LEVEL SECURITY

Page 6: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

tldr;

visibilities are like ACLs

...sort of

CELL-LEVEL SECURITY

Page 7: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THAT’S GREAT!

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What does it get me?

Page 8: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THAT’S GREAT!

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What does it get me?

Amalgamating data sources that are segregated

Page 9: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THE SCENARIO:

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

I am a first time Accumulo userI want to use it’s nifty featuresI have no idea what I’m doing

Page 10: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FIRST TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Scan without JohnsLabel

Page 11: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FIRST TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Scan without JohnsLabel*sad trombone*

Scan with JohnsLabel

Page 12: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FIRST TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Scan without JohnsLabel*sad trombone*

Scan with JohnsLabelrow1 colf1:colq1 JohnsLabelrow1 colf2:colq1 JohnsLabelrow2 colf1:colq3 JohnsLabelrow3 colf1:colq1 JohnsLabelrow4 colf4:colq2 JohnsLabel

Page 13: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SECOND TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row1 colf1:colq1 JohnsApplicationrow1 colf2:colq1 JohnsApplicationrow2 colf1:colq3 JohnsApplicationrow3 colf1:colq1 JohnsApplicationrow4 colf4:colq2 JohnsApplication

Page 14: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SECOND TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What does my label even mean?

row1 colf1:colq1 JohnsApplicationrow1 colf2:colq1 JohnsApplicationrow2 colf1:colq3 JohnsApplicationrow3 colf1:colq1 JohnsApplicationrow4 colf4:colq2 JohnsApplication

Page 15: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THIRD TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row1 colf1:colq1 application1|application2row1 colf2:colq1 application1row2 colf1:colq3 application2row3 colf1:colq1 application2row4 colf4:colq2 application3

Page 16: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THIRD TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What about analytic4?analytic5? 6?

row1 colf1:colq1 application1|application2row1 colf2:colq1 application1row2 colf1:colq3 application2row3 colf1:colq1 application2row4 colf4:colq2 application3

Page 17: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

BACK TO THE DRAWING BOARD

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What am I trying to accomplish?Why am I segregating my data?

Page 18: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FOURTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row1 colf1:colq1 org1|org2row1 colf2:colq1 org1row2 colf1:colq3 org2row3 colf1:colq1 org2

row4 colf4:colq2 org1&org2

Page 19: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FOURTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Organizations are big!

row1 colf1:colq1 org1|org2row1 colf2:colq1 org1row2 colf1:colq3 org2row3 colf1:colq1 org2

row4 colf4:colq2 org1&org2

Page 20: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FIFTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row1 colf1:colq1 subOrg1|subOrg2row1 colf2:colq1 subOrg1row2 colf1:colq3 subOrg2row3 colf1:colq1 subOrg2

row4 colf4:colq2 subOrg1&subOrg2

What about if subOrgs change?

Page 21: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FIFTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What about if subOrgs change?Why do these orgs have permission?

row1 colf1:colq1 subOrg1|subOrg2row1 colf2:colq1 subOrg1row2 colf1:colq3 subOrg2row3 colf1:colq1 subOrg2

row4 colf4:colq2 subOrg1&subOrg2

Page 22: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SIXTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row1 colf1:colq1 accountsReceivable|payrollrow1 colf2:colq1 accountsReceivable

row2 colf1:colq3 payrollrow3 colf1:colq1 payroll

row4 colf4:colq2 accountsReceivable&payroll

Looks good!

Page 23: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SIXTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Looks good!But now I need to manage users!

row1 colf1:colq1 accountsReceivable|payrollrow1 colf2:colq1 accountsReceivable

row2 colf1:colq3 payrollrow3 colf1:colq1 payroll

row4 colf4:colq2 accountsReceivable&payroll

Page 24: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Page 25: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

okay… what is this?

Page 26: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

tserverscan

PluggableAuthorizor

getAuths()scan

Page 27: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

tserverscan

PluggableAuthorizor

getAuths()scan

Now we can use our existing system!

Page 28: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SEVENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

LDAP’s role-based access says:User1->HR

User2->InternalConflictsUser3->PayrollUser4->Taxes

Page 29: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SEVENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

One less system to maintain!

LDAP’s role-based access says:User1->HR

User2->InternalConflictsUser3->PayrollUser4->Taxes

Page 30: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

SEVENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

One less system to maintain!But our orgs are hierarchical!

LDAP’s role-based access says:User1->HR

User2->InternalConflictsUser3->PayrollUser4->Taxes

Page 31: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

EIGHTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Authorizor Says:InternalConflicts->InternalConflicts,HR

Payroll->Payroll,FinanceTaxes->Finance,AccountsReceivable

Page 32: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

EIGHTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

But what if I don’t want a certain org to get a piece of data?

Authorizor Says:InternalConflicts->InternalConflicts,HR

Payroll->Payroll,FinanceTaxes->Finance,AccountsReceivable

Page 33: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What if I don’t want a certain org to get a piece of data?

Page 34: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

NINTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row5 colf1:colq3 designer&!manager

Page 35: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

NINTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Accumulo does not support NOTs

row5 colf1:colq3 designer&!manager

Page 36: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

NINTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Accumulo does not support NOTsWhat are we trying to accomplish?

row5 colf1:colq3 designer&!manager

Page 37: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

TENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row5 colf1:colq3 designer&(worker&contractor)

Page 38: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

TENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

But I want others to know some part of row5 colf1:colq!

row5 colf1:colq3 designer&(worker&contractor)

Page 39: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

REMEMBER

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Page 40: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

ELEVENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row5 colf1:colq3 designer&(worker&contractor)row5 colf1:colq3 engineer&(worker&contractor)

Page 41: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

ELEVENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row5 colf1:colq3 designer&(worker&contractor)row5 colf1:colq3 engineer&(worker&contractor)

But I still want the managers to know that row5 colf1:colq3 exists!

Page 42: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

TWELTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row5 colf1:colq3row5 colf1:colq3 designer&(worker&contractor)row5 colf1:colq3 engineer&(worker&contractor)

Page 43: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

TWELTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

How can root look at everything?

row5 colf1:colq3row5 colf1:colq3 designer&(worker&contractor)row5 colf1:colq3 engineer&(worker&contractor)

Page 44: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THIRTEENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

row5 colf1:colq3row5 colf1:colq3 root|

(designer&(worker&contractor))row5 colf1:colq3 root|

(engineer&(worker&contractor))

Page 45: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THIRTEENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

I don’t like that...

row5 colf1:colq3row5 colf1:colq3 root|

(designer&(worker&contractor))row5 colf1:colq3 root|

(engineer&(worker&contractor))

Page 46: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THIRTEENTH TRY 2

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Remember the pluggable Authorizor!

LDAP knows all rolesroot->all roles

Page 47: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

THIRTEENTH TRY 2

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

All of my bases are covered!

Except...

Remember the pluggable Authorizor!

LDAP knows all rolesroot->all roles

Page 48: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

GETTING CRAFTY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

What if I want to:● Allow authorizations based on time● Allow authorizations based on location● Make data more available● Make data less available

Page 49: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

BEING CRAFTY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Remember the pluggable Authorizor!

If you have the data available, you can use it!

Page 50: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

BEING CRAFTY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Remember the pluggable Authorizor!

If you have the data available, you can use it!

Just remember- visibility labels are filters. They’re not made for restricting

entire tables.

Page 51: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FOURTEENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Accumulo Tables have Read permissions for coarse access!

Page 52: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

FOURTEENTH TRY

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Accumulo Tables have Read permissions for coarse access!

Can we do it to people who are missing certain labels?

Page 53: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Page 54: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Looks familiar… what is this?

Page 55: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

tserverscan

Pluggable PermissionHandler

hasTablePermission()scan

Page 56: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

PLUGGABLE SECURITY TO THE RESCUE

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

tserverscan

Pluggable PermissionHandler

hasTablePermission()scan

Now we can use our existing systemfor coarse access!

Page 57: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

RECAP

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

● Label for the data, not the users● Label with the highest granularity

possible● Let the pluggable security do the rest of

the work● Need to rely on external services or

special processes for tracking labels● These can manage users authorizations

and general access

Page 58: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

RECAP

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Cell level security boils down to two separate components● Data labels● User granted labels

They are the two halves that establish cell level security.

Page 59: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

RECAP

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

Cell level security boils down to two separate components● Data labels● User granted labels

They are the two halves that establish cell level security. Put the two together, and magic happens.

Page 60: Accumulo Summit 2014: Accumulo Visibility Labels and Pluggable Authorization Systems: A Love Story

© 2014 Sqrrl | All Rights Reserved | Proprietary and Confidential

QUESTIONS?

@ohshazbot

[email protected]

ACCUMULO VISIBILITY LABELS AND PLUGGABLE AUTHORIZATION:

A LOVE STORY