Download - Sqrrl March Webinar: How to Build a Big App
Securely explore your data
BUILDING BIG APPS® WITH SQRRL ENTERPRISE
Data-Driven Processes to Making Your Big Data Actionable Joe Travaglini, Director of Product Marketing 3/27/2014
OUTLINE
• Brief Company and Product Overview
• Setting Context: The “Big Promise”
• Steps for Building a Big App
• How Sqrrl can help
2 © 2014 Sqrrl Data, Inc. | All Rights Reserved
COMPANY OVERVIEW
© 2014 Sqrrl Data, Inc. | All Rights Reserved 3
WHO WE ARE
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Sqrrl’s team has worked with the world’s largest and most sensitive datasets for the last decade
" Over 20 years of combined Accumulo experience
" Team includes former Technical Director of Accumulo at NSA and 6 committers/contributors
Google’s BigTable Paper
2006
NSA Builds Accumulo
2008 Sqrrl Founded
2012 1st Sqrrl Release and Customers
2013
NSA Open Sources
Accumulo 2011
Investors:
4
OUR KEY FEATURES
© 2014 Sqrrl Data, Inc. | All Rights Reserved
Sqrrl Enterprise has best-in-class security, scalability, and flexibility
5
Data-Centric Security
Unprecedented Scale and Rates of Ingest
Automated Labeling and Indexing
Secure Full-Text and Graph Search
Column, Document, and Graph Data Models
Query: SqrrlQL Analytics: Real-Time Aggregates and Hadoop Integrations
SETTING CONTEXT
© 2014 Sqrrl Data, Inc. | All Rights Reserved 6
THE “BIG PROMISE”
“We entered the age of Big Data when the opportunity cost of throwing data away exceeded the monetary cost of keeping it…”
… but how do we take advantage of the data to realize the opportunity?
© 2014 Sqrrl Data, Inc. | All Rights Reserved 7
How can we make Big Data fulfill the hype?
A PROPOSED SOLUTION
• Data Lake as one-stop shop for all things data • Great in theory, difficult in practice:
• How do you catalog and search data from different origins?
• What controls are in place for security/compliance? • How do you turn a big filesystem into a useful
application?
• If you don’t have good answers to the above, you are now the proud owner of a Data Landfill
© 2014 Sqrrl Data, Inc. | All Rights Reserved 8
…Enter the Data Lake / Data Hub
KEYS TO SUCCESS
• Robust security
• Easy ways to acquire data
• Easy ways to retrieve data
• Compelling ways to interact with data
© 2014 Sqrrl Data, Inc. | All Rights Reserved 9
Our Core Values and Areas of Focus
BUILDING A BIG APP
© 2014 Sqrrl Data, Inc. | All Rights Reserved 10
OK then…
BUILDING A BIG APP
1. Define the business objective ! Spell out your goals, outcomes, metrics, KPIs
2. Define users and roles ! What are the classifications of users of your app? ! What actions do they need to perform?
" Strive for specificity
3. Identify relevant datasets ! Specify partial schemas where appropriate ! Consider which fields are most important to index ! Consider data volume for proper capacitization
© 2014 Sqrrl Data, Inc. | All Rights Reserved 11
Our Methodology for tackling Big Data Problems
TUTORIAL
© 2014 Sqrrl Data, Inc. | All Rights Reserved 12
Dataset Creation in Sqrrl
BUILDING A BIG APP, CON’T
4. Determine access policies for the data ! Granular Cell-Level Security Enforcement ! Attribute-based Access Control
5. Load the Data ! Access Labeling and Indexing
© 2014 Sqrrl Data, Inc. | All Rights Reserved 13
Our Methodology for tackling Big Data Problems
TUTORIAL
• Rules file:
• Bulk import:
© 2014 Sqrrl Data, Inc. | All Rights Reserved 14
Bulk Loading in Sqrrl
TUTORIAL
Note that username and bytesUp are not visible to this user!
© 2014 Sqrrl Data, Inc. | All Rights Reserved 15
Bulk Loading Results
BUILDING A BIG APP, CON’T
4. Determine access policies for the data ! Granular Cell-Level Security Enforcement ! Attribute-based Access Control
5. Load the Data ! Access Labeling and Indexing
6. Transform the Data ! More often than not
7. Syndicate the Data ! Use the Sqrrl Thrift API to wire a front-end ! Export the data to HDFS, downstream systems
© 2014 Sqrrl Data, Inc. | All Rights Reserved 16
Our Methodology for tackling Big Data Problems
BUILDING A BIG APP, CON’T
8. Peruse the data ! Our query language: SqrrlQL
" Real-time, Secure Text Search " Graph Search " Summaries / Filters
9. Evaluate and Refine ! Did I answer the questions I wanted? ! Did the results match expectations? ! Are there new questions to ask?
10. Rinse and Repeat
© 2014 Sqrrl Data, Inc. | All Rights Reserved 17
Our Methodology for tackling Big Data Problems
RECAP
• Start your journey with specific goals in mind
• Big Data can get bigger – and usually does • Data-Centric Security enables comfortable
consolidation and integration • Real-time search, discovery, and analytics are the
core of Big Apps
• Sqrrl is here to help!
© 2014 Sqrrl Data, Inc. | All Rights Reserved 18
WHAT’S NEXT
April’s Webinar: Healthcare Apps
with Medyear by Personiform
powered by Sqrrl Enterprise
© 2014 Sqrrl Data, Inc. | All Rights Reserved 19
THANKS!
FOR MORE INFO: • Visit us at http://www.sqrrl.com • Follow us on Twitter -- @SqrrlData • Send us an email: [email protected]
Presented by: JOE TRAVAGLINI Director of Product Marketing [email protected] @joe_travaglini
© 2014 Sqrrl Data, Inc. | All Rights Reserved 20