using open source bi in the real world

Open Source

Adoption and Use in the Real World

0

20

40

60

80

1001

2

3

4

5

6

789

10

11

12

13

14

15

Margin of error: +/- 100 points

Smooth

Chunky

Summing up data warehousing & business intelligence:

Transaction processing is a commodity. Analysis is not.

Cautionary Tale: Visions of Yesterday’s Future

“Prediction is very difficult, especially about the future.”

Niels Bohr

“Open source is not worth paying attention to.”A Gartner analyst I don’t want to make too much fun of, January 2006

Where the analysts are on the adoption curve

“The future is the present projected.”Aldous Huxley

What is the state of the enterprise software market today?

March 2009 Mark R. Madsen

Any Industry This Big is Maturing

Annual US software sales

-10

10

30

50

70

90

110

130

150

70 75 80 85 90 95 00Source: US Dept. of Commerce


Evolution of the Software Market 1987

Source: John Prendergast (data: Bloomberg, Factset)


The DW & BI Software Market Today According to IDC, the analytics and data warehouse software market is growing at 10.3% CAGR

17,38619,342

21,40823,601

26,00128,682

31,595

2005 2006 2007 2008 2009 2010 2011

How to predict like an analyst.

Moore’s Law via the Lens of the Industry Analyst

Time

CPU Speed

Moore’s Law: Power Consumption

Time

Power Use

2019

Moore’s Law: Heat Generation

Time

CPU Temp

2019

Conclusion #1: Your own nuclear reactor by 2019

Time

Power Use

2019

Conclusion #2: You will need a new desk in 2019

Time

CPU Temp

2019

“If the automobile had followed the same development as the computer, a Rolls-Royce would today cost $100, get a million miles per gallon, and explode once a year killing everyone inside.”

Robert Cringely

Time

Anything

Reality


The Real State of Enterprise Software?


Software Revenue = Corporate IT Cost

IT costs as a percent of equipment investment

0

10

20

30

40

50

68 72 76 80 84 88 92 96 00 04Source: US Dept. of Commerce


Enterprise Software Economics

! 70% - 80% of sales & marketing is for new sales

! 76% of new license revenue goes to sales & marketing

! Maintenance makes up 45% of revenues and this number is increasing

! 75% of R&D for mature products is for updates, bug fixing, and non-revenue enhancements

! Maintenance and support is becoming the biggest factor is software company profitability.

Sources Godman-Sachs, Tech Strategy Partners, Forrester

The enterprise software model is breaking down. Some facts:


BI is Entering Mainstream Adoption

This means the BI market is entering a period of commodification: demand up, supply up, prices and margins down. Door open for OSS.

Platforms

DatabasesReporting & Analysis

Data Integration

Predictive analytics


Technology Priorities in IT

Informing the business trumps automating the business.This held true for three years in a row.

Source: CIO Insight


Spending Priorities in IT

In 2007 and 2008 IT budgeted most new project money for databases and business intelligence.

Sources: CIO Insight


Open Source Disruption

“Which sector of the industry is most vulnerable to disruption by open source in the next five years?”

1. Web publishing and content management2. Social software3. Business Intelligence

Source: North Bridge Venture Partners


Signs of Maturity

Source: Open Source Index 2008, Red Hat, Inc.


Use of OSS BI/OLAP tools worldwide


Open Source BI Use Looks Like Proprietary BI Use


Rationale When Evaluating OSS


Good News: It Works

State of Adoption & Use of Open Source BI

33%

9%12%

18%21%

0%5%

10%15%20%25%30%35%40%45%50%

Database!/DW!platform

Reportingand!OLAP

Dataintegrationand!ETL

Embedded!/applicationreports

Advancedanalytics

None Considering Completed!Evaluation Using!in!Production

45%

22%

7% 7%9%

7%3%

0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

0"49 50"100 100"499 500"999 1"5!TB 5"25!TB >25!TB

81% of the sample < 1TB

Data size for all survey respondents including those using proprietary databases.


Why did BI software evaluations fail?

25%

26%

27%

41%

56%

0% 10% 20% 30% 40% 50% 60%

Required!more!expertise!than!expected

Difficulty!integrating!into!environment

Lack!of!available!consulting

Scalability!problems

Missing!or!incomplete!features

There’s still work to be done

Data is the future


Questions?“When a new technology rolls over you, you're either part of the steamroller or part of the road.” – Stewart Brand


We Could Use Your HelpIf you evaluated open source software for any aspect of the BI or data warehouse environment, please fill out the online open source adoption survey athttp://bitly.com/scRhF

The survey is running until May 30, 2009.


Creative CommonsThanks to the people who made their images available via creative commons:highway storm.jpg - http://flickr.com/photos/areyoumyrik/235230688firemen not noticing fire.jpg - http://flickr.com/photos/oldonliner/1485881035/godzilla_vs_bhudda_big.jpg - http://flickr.com/photos/olivander/262293544/acapluco_cliff_divers_cc.jpg - http://flickr.com/photos/raveller/


Creative CommonsThis work is licensed under the Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 United States License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/3.0/us/ or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.

‘

MySQL Conference & ExpoBruce Belvin

Company Mission

Monolith Software Solutions is dedicated to providing scalable business intelligence for multi-unit QSR restaurant operations.

Open Source Components

SUSE Linux Enterprise

Overview

! 3000+ disparate data sources! 4500 users! Complex organizational structures / hierarchy! Multi tenant environment! Segregated data bases per individual organization! Same data used for various business functions! Granular data

Vertical Landscape

I. Fragmented ownershipII. Legacy hardware/various data sourcesIII. Hesitancy to adopt Open SourceIV. Small margin industry

Why SaaS Works

• Subscription business model fits segment price pressures

• Unlimited users solves user heavy structure• Initial price / on going maintenance• Low barrier to entry• Pay as you go for additional integration/modules

Keys to SaaS Success

• Educate multiple decision making groups within organization

• Utilize support from technology partners and open source community

• Be aware of impact on IT/political past decisions

• Prove open source solution

• Develop silver bullet strategies to over come open source perceptions

Background

" President / COO of Consorte Media

" Formerly CTO of BlueLithium, Adteractive, Fathom Online, and Cybernautics

" 13 years as a technical executive in the online advertising industry

54

Scope of Online Advertising

55

Delivers the right ad to the right

person

Collect Metrics for performance

measurement and analytics

Dynamically builds pages for

visitor using predictive models

Business Challenges

56

Performance Reports

Internal Applications

Revised Models

Web API

Analytics

Data Mining

Model Development

Our Stack

" CentOS

" MySQL

" BIRT

" Hibernate

" Apache

" Camel

" Kettle

" Hadoop

Best Practices

58

• Use analytics to design and test advertising models using only

relevant dimensions

• Gather and determine business requirements before embarking

on the journey

• Build an infrastructure plan that will support the data collection

and analytics platform

The Role of Open Source

59

• Several important innovations in data processing have been driven largely by

online advertising

• Industry needs software and tools to match pace of innovation and fast-

changing business climate

• Proprietary software vendors unable to respond quickly enough to support the

industry

• Open Source has provided innovative solutions and flexibility to support new

business requirements

Jay WebsterPresident and COO

[email protected] ext 248

Q&A: Bruce Belvin, Jay Webster, Mark Madsen


We Could Use Your HelpIf you evaluated open source software for any aspect of the BI or data warehouse environment, please fill out the online open source adoption survey athttp://bitly.com/scRhF

The survey is running until May 30, 2009.

using open source bi in the real world

Technology

data warehouse environment

open source bi

open source

business intelligence

creative commons

march 2009 mark

march 2009 mark

john prendergast