seeing redshift: how amazon changed data warehousing forever

The Briefing Room

Twitter Tag: #briefr

The Briefing Room

Welcome

Host: Eric Kavanagh

[email protected]


The Briefing Room

!   Reveal the essential characteristics of enterprise software, good and bad

!   Provide a forum for detailed analysis of today’s innovative technologies

!   Give vendors a chance to explain their product to savvy analysts

!   Allow audience members to pose serious questions... and get answers!

Mission


The Briefing Room

APRIL: Intelligence

May: INTEGRATION

June: DATABASE

July: CLOUD


The Briefing Room

Intelligence

Processing Monitoring Alerts/triggers/actions If it’s not accessible, it’s not achievable

COST

COMPLEXITY

PERFORMANCE

BARRIERS


The Briefing Room

Analyst: Claudia Imhoff

Claudia Imhoff is the CEO of Intelligent Solutions


The Briefing Room

Birst

! Birst offers a SaaS-based, multi-tenant BI platform; it can also be deployed on-premise

!   The Birst solution is capable of unifying siloed technologies, automating data management and providing agile enterprise-class analytics

! Birst’s approach enables self-service analytics by allowing business users to manage and add new data sources, create custom dashboards and collaborate across the organization


The Briefing Room

Brad Peters

Brad Peters is the CEO and co-founder of Birst. Brad has spent the last 10 years building analytics products and solutions. Prior to working at Birst, he helped found and later led the Analytics product line at Siebel Systems, which forms the basis of Oracle’s current OBIEE product family. Brad started his career as an investment banker for Morgan Stanley in the New York M&A practice. Brad regularly blogs for Forbes.com where he writes about Cloud and business software related issues.

AMAZON REDSHIFT & BIRST A NATURAL FIT

AMAZON REDSHIFT: A FRACTION OF TRADITIONAL COMPUTING COSTS

•  Compare $1,000/TB per year to on-‐premise data warehouse •  Ini%al: RDBMS license + Hardware + DW Development

•  Ongoing: Maintenance + Staffing

•  Commodity Map Reduce only 20% less •  OpPon to put into something significantly

more queryable is compelling

10

“The average )me for the construc)on of a data warehouse is 12 to 36 months and the average cost for its implementa)on is between $1 million to $1.5 million.”

Noumenal Consul%ng September, 2010

Faster, Simpler – More Agile Big Data AnalyPcs • Op%mize data driven decisions

• Automate the data transforma%on tasks

• Enable business folks to do what they do best • Answer business ques%ons inside of data

FROM DATA TO ANSWERS

Faster, Simpler – More Agile Big Data Engine • Op%mize Query and I/O

• Automate the data administra%ve tasks

• Enable data stewards to do what they do best •  Ensure data is accessible, performant and secure

11

AMAZON REDSHIFT: BRINGING BIG DATA TO BUSINESS RelaPonal database: business analyst • Flexibility to bring different data types/sources together • Complex dimensional queries – on the fly

MapReduce, Hadoop: data scienPst • Complex to fully leverage data • HiveQL & Hadoop-‐only tools limited

• GeXng beyond simple aggrega%ons is painful/not possible

• Batch process makes broad access untenable

12

BIRST: GIVING BUSINESS MEANING TO YOUR BIG DATA

Examples:

13

•  “as-‐is” vs. “as-‐was” •  Common/conformed dimensions

•  Sophis%cated hierarchies •  Cross data source metrics

• Many-‐to-‐many rela%onships

• Mul%-‐pass / Mul%-‐level ques%ons

Must do two things: 1. Organize the data for rich ques%ons •  Business metrics •  Dimensional analysis

2. Enable business users to ask rich ques%ons •  Interac%ve, ad hoc capabili%es •  Logical layer

BIRST: GIVING BUSINESS MEANING TO YOUR BIG DATA

14

Must do two things: 1. Organize the data for rich ques%ons •  Business metrics •  Dimensional analysis

2. Enable business users to ask rich ques%ons •  Interac%ve, ad hoc capabili%es •  Logical layer

BIRST: THE ONLY END-‐TO-‐END SOLUTION FOR AMAZON REDSHIFT

Connect to Source

ApplicaPons

Automated Data Warehouse

Automated Data Model Logical Layer

De-‐normalize Data

Create Dimensional

Model

Create Business Model

Distribute Insight

Only parPal support by VisualizaPon, Dashboard-‐only, and other Discovery Tools

OLAP BI Tools (e.g. SAP Business Objects, Microstrategy, Oracle BI, IBM Cognos)

ConvenPonal AnalyPcal ETL tools (e.g. InformaPca, etc.)

WAREHOUSE AUTOMATION

Step 1: Denormalize and cleanse Step 2: Map into dimensional model

16

17

Finance Data CRM Data Opera%ons Data More Data

DW

Sandbox Sandbox

Dashboards

Ad Hoc

Reports

Unified Logical Model

ODS

Users

LEVERAGING THE POWER OF REDSHIFT

FROM DATA TO ANSWERS -‐ IN THE CLOUD Why pull data out of Amazon Redshia? • Moving data across the cloud is more expensive and slow than manipula%ng it in place

Leverage the power of Amazon Redshia with ELT • Meaning: Manipulate the data IN THE DATABASE

Reap benefits of mulP-‐tenant analysis • Mul%ple projects, mul%ple user communi%es, one shared infrastructure

18

390,000,000

3,100,000

560,000

420,000

32,000

4,000

ENTERPRISE CALIBER BI BORN IN THE CLOUD

MB of Data

Dashboards

Dimension Tables

Fact Tables

Dashboard views a day

Organizations

20

LEADERS RELY ON BIRST En

terprise

Clou

d Mid-‐m

arket

ABOUT BIRST

• #1 Cloud BI Provider Market & Product Leader • More than 1,000 organiza%ons rely on Birst •  Founded in 2005

21

“ No. 1 in product func0onality and customer (that is, product quality, no problems with so=ware, support) and sales experience.”

2013 Business Intelligence Magic Quadrant Challenger

DEMONSTRATION

LEARN MORE

Join us for a Live Demo • Every Tuesday and Thursday at 11:00 am PT/2:00 pm ET

• Register at birst.com/livedemo

Try Birst with Birst Express • birst.com/express

Contact us • Email: [email protected] • Phone: (866) 940-‐1496


The Briefing Room

Analyst: Claudia Imhoff

Perceptions & Questions

Copyright © 2013, Intelligent Solutions, Inc., All Rights Reserved

Claudia Imhoff

President Intelligent Solutions, Inc.

Founder Boulder BI Brain Trust (BBBT)

A thought leader, visionary, and practitioner, Claudia Imhoff, Ph.D., is an internationally recognized expert on analytics, business intelligence, and the infrastructures to support these initiatives. Dr. Imhoff has co-authored five books on these subjects and writes articles (totaling more than 100) for technical and business magazines. She is also the Founder of the Boulder BI Brain Trust (www.BoulderBIBrainTrust.org), a consortium of independent analysts and consultants. You can follow them on Twitter at #BBBT.

Email: [email protected] Phone: 303-444-6650 Twitter: Claudia_Imhoff

25


General Cloud BI Advantages

§  Low-cost, low-risk, low-maintenance and fast development §  Usage-based billing and predictable monthly costs §  On-demand capacity – easy to deploy, grow & shrink users §  Secure and high availability §  New product features delivered rapidly §  Can also be used for developing in-house solutions §  Vendors support only one platform / one version of app §  Cloud BI model gives vendor a predictable cash flow

26


General Cloud BI Disadvantages

§  Cloud model produces less upfront vendor revenue but higher customer set up costs

§  May lead to stovepipe Cloud systems with limited controls §  May involve complex integration with existing systems §  May involve complex customization and tuning for large

projects §  Customers still need ability to integrate Cloud application

data with other enterprise data

27


Enter Redshift

§  Fast, fully managed, petabyte-scale DW service §  Optimized for datasets from few 100 gigabytes to a

petabyte or more §  Delivers fast query and I/O performance using columnar

storage technology (ParAccel) §  Automated most common admin tasks around provisioning,

configuring, monitoring, back-ups, and security §  Pricing is simple – an hourly rate based on node type and

number of nodes in a cluster – no upfront prices §  Compatible with industry standard ODBC and JDBC

connections and Postgres drivers

28


Good News About Redshift

§  Now have ability to provision huge database volumes §  No long, protracted procurement process to get HW/SW

and no maintenance cost §  Ability to grow as you do – perhaps beyond petabytes! §  Potentially huge cost savings over years versus cost of

own HW/SW §  Great elasticity in terms of adding/subtracting users §  Great performance for complex analytics – return results

very quickly

29


Things to Think About with Redshift

§  Possibility of an outage – it’s happened before – need service-level agreements

§  Costs of data migration and integration – you need massive bandwidth to transmit data or lots of USB drives

§  Very new processes – no established best practices yet (but Amazon has a very thorough “getting Started Guide)

§  Potentially higher costs than on-premises over time §  Per user pricing can become expensive for large numbers §  You pay for all the data whether you use it or not

30


Birst on Redshift

§  Birst has all the necessary components for BI solution §  ETL, semantic layer of business terms, multiple deployment

methods (dashboards, reports, mobile devices)

§  No DBAs required to create tables and write load scripts (scripts are generated by Birst)

§  Tight integration with Redshift means fast data processing – maximized speed, scale and performance §  Analytic results returned quickly so buisness can act quickly

My bottom line: Redshift and Birst gives traditional

data warehousing players a run for their money.

31


Questions

§  What suggestions do you have for your customers to mitigate or eliminate the potential for “silos” of data—integrating their on-premises data warehousing system and data now in cloud deployments?

§  What have been the significant benefits your customers have received from moving to the Redshift offering?

§  What are the realistic deployment times for a Redshift + Birst implementation?

§  Still a question today is “Will organizations trust in a cloud solution for critical analytics?” How do you answer that?

§  Every company likes to think of itself as unique. How do you accommodate this uniqueness in a cloud-based solution (customization capabilities)?

32


Questions

§  What do you say to the bandwidth problem? §  Do you have best practices for new customers moving to Redshift and

Birst? What are they? §  When does it not make sense for a company to move to the Redshift +

Birst combination but stay with an on-premises deployment? §  How easy will it be to move from Redshift back to an on-premises

version? What would be the reasons for such a shift? §  What do you see for the future of your partnership with Amazon?

33


The Briefing Room


The Briefing Room

April: INTELLIGENCE

May: INTEGRATION

June: DATABASE

Upcoming Topics

www.insideanalysis.com


The Briefing Room

Thank You for Your

Attention

Certain images and/or photos in this presentation are the copyrighted property of 123RF Limited, their Contributors or Licensed Partners and are being used with permission under license. These images and/or photos may not be copied or downloaded without permission from 123RF Limited.

seeing redshift: how amazon changed data warehousing forever

Technology

data management

data administra

data transforma

data stewards

big data exampl

briefing room birst

briefing room twitter

data scienpst complex