bi survey 12 results updated -...

Free Preview

lk

THE

BI SURVEY 11 SURVEY

The Customer Verdict The world’s largest survey of business intelligence software users

1211

A preview of The BI Survey 12: The Results For more information, visit: www.BI-Survey.com

This document is not to be shared, distributed or reproduced in any way without first purchasing licensing rights and the express prior permission of BARC.

http://www.bi-survey.com/

The BI Survey 12 – The Results

2

Table of Contents

Introduction ............................................................................................................................ 4

Business Benefits and the BBI ............................................................................................... 7

BBI by peer group and vendor ........................................................................................... 9

Goal Achievement ................................................................................................................ 17

Deployment .......................................................................................................................... 22

Usage in different company size classes ......................................................................... 22

Number of BI users .......................................................................................................... 24

BI usage by departments ................................................................................................. 25

Product usage for application scenarios .......................................................................... 29

The selection process .......................................................................................................... 33

Evaluation methods ......................................................................................................... 33

Reasons to buy ................................................................................................................ 37

How products perform in the selection process ................................................................ 42

Buying BI software ............................................................................................................... 45

Licenses .......................................................................................................................... 45

Number of users .............................................................................................................. 47

Cost ................................................................................................................................. 48

Implementing BI and Satisfaction ......................................................................................... 54

Implementation time ........................................................................................................ 54

Project scope ................................................................................................................... 58

Support ............................................................................................................................ 61

Recommendation ............................................................................................................. 62

Chosen as standard ......................................................................................................... 65

Problems in BI projects .................................................................................................... 67

Query Performance and Data Volume .................................................................................. 69

Query Performance results and user satisfaction ............................................................. 69

Data volumes handled per product .................................................................................. 73


3

Front ends for multidimensional databases .......................................................................... 75

Infor ION BI OLAP Server ................................................................................................ 76

Microsoft SSAS ............................................................................................................... 76

Oracle Essbase ............................................................................................................... 78

IBM Cognos TM1 ............................................................................................................. 79

SAP BW .......................................................................................................................... 80

Trending topics in BI ............................................................................................................. 82

Three-step approach to analyzing trends ......................................................................... 82

The frequent inaccuracy of plans ..................................................................................... 83

User perception of IT and software trends ....................................................................... 84

Status and plans for trending topics in BI ......................................................................... 87

Visual Analysis & Data Discovery .................................................................................... 88

Collaboration ................................................................................................................... 90

Mobile BI ......................................................................................................................... 91

Cloud BI/BIaaS ................................................................................................................ 94


4

Introduction

This document provides an overview of the key product-related results from The BI

Survey 12. The Survey provides a detailed quantitative analysis of why customers

buy BI tools, what they are used for, how successful they are and why they

eventually abandon them. This edition has increased the range of products

reviewed, including not only products from well-known BI giants, but also specialist

tools from much smaller vendors and open source vendors.

The BI Survey 12 follows on from ten successful editions of The BI Survey (formerly

The OLAP Survey). It is important to note that this year the numbering of The BI

Survey has changed so that The Survey number now aligns to the year in which the

data was collected.

Based on analysis of real-world experiences of nearly 3,000 respondents, the value

of The Survey is dependent on a sufficiently large, well-distributed and unbiased

sample. The Survey is the largest and most thorough fact-based analysis of the BI

market currently available, using eleven years of experience to analyze market

trends and debunk many myths surrounding the BI industry.

After data cleansing and the removal of responses from participants unable to

answer specific questions on BI tools, we were left with a sample of 1872 end users,

283 consultants and 512 vendor employees. Participants from all over the world took

part in The BI Survey 12. Around 20 percent of respondents had an IT job function,

15 percent were Heads of BI, and 65 percent had various lines of business job titles.

The BI Survey is not based on anecdotal accounts or personal opinions, unlike much

analyst research. The Survey is not intended to be a measure of market shares and

unlike our research publication, ‘The BI Verdict’, it does not include product reviews

or qualitative product comparisons. The BI Survey does not attempt to forecast

future trends, instead providing evidence undermining the reliability of many such

forecasts.

In the past the vast quantity of data collected by The Survey has been presented in

one large document. However, The BI Survey 12 heralds the introduction of several

bitesize documents, each focusing on a specific area of The Survey.


5

Detailed information on the sample and Survey methodology can be found in the

‘Sample, Products and Methodology’ document. For advice on selecting, managing

and implementing products into your organization see the document ‘The BI Survey

12: Best Practices’.

Document Description

The BI Survey 12 - The Results Provides an overview and analysis of the most important results of The BI Survey 12

The BI Survey 12 – Trending Topics series

Provides in-depth analysis of the following trending topics: SaaS/Cloud BI, Mobile BI, Big Data, Social BI, Collaboration, Self Service, Analytical Databases

The BI Survey 12 – Vendor Performance Summaries

A series of short executive reports highlighting all the product-related results for each vendor.

The BI Survey 12 – Best Practices

This report features recommendations on how best to choose a BI product and the most important and relevant tool selection criteria.

The BI Survey 12 - Sample, Products, Methodology (this document)

Provides details of the sample and an overview of our methodology including the entire questionnaire and details of our calculation methods.

The BI Survey 12 - KPIs and Dashboards

This document outlines the definitions and calculation methods of the KPIs used in The BI Survey 12.

Figure 1: Overview of The BI Survey 12

The BI Survey Analyzer is an online tool allowing users to carry out their own

analyses of The Survey data. Using results from this year’s Survey, the tool also

offers users the ability to drill down using product and demographic filters.

2012 also sees the exciting introduction of a mobile version of The BI Survey

Analyzer for the iPad. The mobile BI Survey Analyzer focuses on product-related

results based on criteria (KPIs) such as business benefits achieved, implementation

costs, functional usage, product competitiveness, innovation, performance, customer

satisfaction and agility.


6

Figure 2: Screenshot of The BI Survey Analyzer

Figure 3: Screenshot of The BI Survey Analyzer iPad App


7

Business Benefits and the BBI

The BI Survey asks key questions regarding the benefits of BI projects. Respondents

are asked to indicate the level of achievement gained from a list of eleven potential

benefits. A scoring system is then used to derive a composite weighted score for

each benefit, based on the level of benefit achieved. This system is called the BBI

(Business Benefits Index). For further information on the calculation methods used,

see the ‘Sample, Products and Methodology’ document.

Figure 4 shows the overall breakdown of responses to the Business Benefits

questions. The six levels of achievement, along with their weightings, are shown on

the horizontal axis. Results are RAG (red, amber, green) rated with green indicating

higher scores and red denoting lower scores.

Figure 4: Frequency of Business Benefi ts (n=2116)

Pro

ven

and

quan

tifie

d (4

961)

Pro

ven,

but

not

m

easu

red

(564

3)

For

mal

ly

clai

med

, bu

t not

ve

rifie

d (2

186)

Inf

orm

ally

su

spec

ted

(274

1)

Not

ach

ieve

d (2

495)

Got

wor

se/m

ore

expe

nsiv

e (2

26)

Do

not k

now

(5

024)

Weighting 10 8 5 3 -2 -6 0

Better business decisions 27.9% 39.5% 12.6% 10.7% 1.8% 0.2% 7.4%

Faster reporting, analysis or planning 45.2% 35.5% 7.4% 5.0% 2.0% 0.3% 4.6%

Improved customer satisfaction 17.7% 29.0% 13.8% 13.8% 5.3% 0.3% 20.1%

Improved data quality 29.6% 32.2% 10.6% 10.5% 6.2% 0.5% 10.3%

Improved employee satisfaction 22.8% 33.9% 13.5% 13.7% 5.3% 0.7% 10.1%

Increased revenues 10.3% 13.3% 11.0% 19.2% 9.0% 0.2% 36.9%

More accurate reporting, analysis or planning 33.6% 38.9% 8.9% 8.7% 1.9% 0.4% 7.6%

Reduced external IT costs 16.6% 14.2% 7.5% 11.8% 14.7% 3.4% 31.7%

Saved business headcount 11.4% 9.5% 5.9% 13.2% 25.7% 0.9% 33.4%

Saved IT headcount 11.1% 9.2% 4.9% 9.0% 29.3% 2.9% 33.6%

Saved other non-IT costs 8.3% 11.4% 7.2% 13.9% 16.7% 0.8% 41.7%


8

Figure 5 shows business benefits listed in order of the most commonly achieved.

Figure 5: Benefits overview (n= 2116)

Figure 5 Key Findings:

� Overall, ‘Faster Business Intelligence’ rated higher than accuracy and more

than 80 percent of respondents stated that faster reporting, analysis and

planning had been achieved. 72 percent reported that ‘More accurate

reporting, analysis and planning’ had been proven and 67 percent of

respondents, up from 59 percent last year, said that ‘Better business

decisions’ was another benefit likely to be achieved. The ability to make

better business decisions is a highly desirable benefit. However, it is a benefit

that cannot be accurately scoped when developing the business case for a

project. While all BI projects would hope to gain this benefit, few projects

would be cost-justified against the possibility that it ‘might’ one day be

achieved.

� ‘Improved data quality’ is a benefit that can be measured directly. Although

often one of the most difficult factors to achieve in a business intelligence

project, it ranked fourth in the list with ‘Employee satisfaction’ following in fifth

place, a benefit that typically arises when users have fast access to data and

reports.

� Benefits with the lowest level of achievement were ‘Saving headcount in IT

departments’, ‘Saving headcount in business departments’, ‘Saving non-IT

costs’ and ‘Reducing external IT costs’. This is in line with our experience that

companies are still looking to extend their BI capabilities, and are adding

resources to implement and run them. If BI were a mature and saturated

market we would see more investment in replacing existing systems but until

1.60

2.02

2.14

3.03

3.03

5.06

5.93

6.23

6.85

7.12

7.82

0 2 4 6 8 10

Saved IT headcount

Saved business headcount

Saved other non-IT costs

Reduced external IT costs

Increased revenues

Improved customer satisfaction

Improved employee satisfaction

Improved data quality

Better business decisions

More accurate Business Intelligence

Faster Business Intelligence


9

this pattern changes we would not expect savings in headcount and costs to

move up the list of benefits.

In contrast to previous results, this year the overall BBI decreased slightly from 4.89

to 4.62. However, we do not think this the beginning of a long-term trend of declining

benefits from BI projects. Instead we suspect that this small variation can be

accounted for by the fact that The Survey sample varies from year to year.

BBI by peer group and vendor

The difference between business benefit achievement according to the type of

product in use is reflected in the following chart. In order to compare product types

we group products into specific peer groups as explained in the ‘Sample, Products

and Methodologies’ document.

Figure 6: Business benefits analyzed by peer group (n=changing bases)

Visual Analysis & Data Discovery tools achieved the best BBI results reflecting their

user friendliness and flexibility in allowing end-users to quickly load data, perform

user analysis and take advantage of advanced visualization features. The user-

friendly nature of these tools relates to business benefits achieved. Small and

Medium Project vendors rank second, Dashboard vendors rank third and

Performance Management products rank fourth. This is a result we have consistently

observed in previous editions of The BI Survey. Specialists, or smaller vendors, tend

to show better BBI results than the larger BI Giants, Large Project vendors and

Enterprise Reporting vendors. There is a straight correlation between the size of the

BI project (measured in data volume used or number of users served) and the size of

the tool vendor. With few exceptions, smaller vendors serve smaller projects and

products from larger vendors are used in larger projects.

4.10

4.134.24

4.41

4.504.61

4.77

4.805.06

0 2 4 6

Enterprise ReportingLarge Project Vendor

BI GiantsIT Giants

OLAP AnalysisPerformance Management

DashboardSmall & Medium Project Vendor

Visual Analysis & Data Discovery


10

In our opinion, there are several reasons for the good results achieved by smaller

and specialist vendors:

� Some customers may achieve more business benefits when working with

smaller vendors with whom they have a close working relationship, compared

to the more arm’s-length, impersonal relationship that users tend to have with

large vendors. Survey results show that users usually experience better

vendor support, satisfaction and recommendation with smaller vendors than

with many of the large vendors.

� The prospect of smaller vendors being removed from a company more

quickly if projects are not successful leads to better than average results for

the ones that remain in use. Conversely, large vendors are able to keep

customers with whom they have long-term, strategic relationships, even if

individual projects or products deliver disappointing results.

� Products from small vendors are more likely to be selected in competitive

evaluations. Users expending more effort selecting solutions show a higher

level of benefit achievement. This could be because the extra diligence taken

to ensure that the product fits their needs increases the probability of the

project delivering business benefits. There is also the simple fact that some

projects are managed better than others.

But, for whatever reason, it is clear that the customers of small vendors such as

Yellowfin, Dimensional Insight and Phocas are reporting more business benefits than

those of larger vendors like SAP, Oracle, Microsoft and IBM. That said, several

products from large vendors have above average BBI scores, most notably the

multidimensional databases Oracle Essbase, IBM Cognos TM1 and Microsoft SSAS.


11

Figure 7 shows the business benefits scores for all products in the BI Giants peer

group.

Figure 7: Business benefits for the BI Giants peer group (n=949)

The BI Giants peer group includes companies with annual revenues of more than

$200 million and a truly international reach. These tools are usually used in

enterprise scenarios.


� The multidimensional database Oracle Essbase ranked first in the BI Giants

peer group. All five top ranking products offer in-memory database

technology, while the traditional ROLAP tools from SAP, Oracle and IBM

deliver the least business benefits.

� Three of the four multidimensional databases in this peer group (all except

Infor) appear in the top five, including QlikTech (with its own in-memory

database technology) and MicroStrategy, having recently expanded its

caching technology.

� The correlation between user self service and fast query performance with

business benefits has been evident in previous BI Surveys.

3.333.45

3.553.76

4.114.23

4.284.50

4.564.64

4.654.904.92

0 2 4 6

SAP BO WebIOracle OBIEE

SAP BWIBM Cognos BI

InforMicrosoft Excel

Microsoft SSRSInformation Builders

MicroStrategyMicrosoft SSAS

QlikTechIBM Cognos TM1

Oracle Essbase


12

Figure 8: Business benefits for the Enterprise Repo rting peer group (n=555)

The Enterprise Reporting peer group includes products that can provide standard

formatted reporting in a large scale enterprise situation. Arcplan, a very flexible tool

for designing reporting applications, ranks highest in this peer group, followed by the

specialist vendors MicroStrategy and Information Builders. Users of the large

vendors SAP, Oracle and IBM report lower business benefits.

Figure 9: Business benefits for the Dashboard vendo r peer group (n=634)

The Dashboard peer group includes products that are focused on creating advanced

dashboards.

Users of tools from small vendors report the highest business benefit scores in the

Dashboard peer group. This group is led by Yellowfin, a tool with a modern

3.33

3.45

3.55

3.76

4.28

4.50

4.56

5.24

0 2 4 6

SAP BO WebI

Oracle OBIEE

SAP BW

IBM Cognos BI

Microsoft SSRS

Information Builders

MicroStrategy

Arcplan

3.45

3.58

4.50

4.56

4.65

5.19

5.24

5.52

6.31

0 2 4 6 8

Oracle OBIEE

Decisyon


MicroStrategy

QlikTech

BOARD

Arcplan

Dimensional Insight

Yellowfin


13

architecture and look and feel, with most of its customers being small companies.

Dimensional Insight, another specialist, comes in second, followed by two more

small vendors: Arcplan and Board.

Figure 10: Business benefits for the OLAP Analysis peer group (n=939)

The OLAP Analysis peer group includes products that support analysis in

dimensional and hierarchical data models. This peer group is crowded and highly

competitive with many vendors scoring good business benefits results. The peer

group is led by Board, followed by three products with very similar results: Targit,

Oracle Essbase and IBM Cognos TM1. As in all previous editions of The BI Survey,

SAP BW users report considerably lower business benefit achievement than any

other product’s users.

3.55

4.11

4.11

4.23

4.35

4.49

4.52

4.56

4.64

4.73

4.90

4.92

4.94

5.19

0 2 4 6

SAP BW

Infor

Pentaho

Microsoft Excel

Evidanza

Cubeware

Jedox

MicroStrategy

Microsoft SSAS

Bissantz

IBM Cognos TM1

Oracle Essbase

TARGIT

BOARD


14

Figure 11: Business benefits for the Visual Analysi s & Data Discovery peer group (n=532)

The Visual Analysis & Data Discovery peer group includes products that provide

advanced visualization features. Dimensional Insight rates top in this small peer

group of specialist vendors, followed by Phocas and Tableau. QlikTech, one of the

most successful BI vendors over the last five years and now considered to be a BI

giant, is starting to show similar results to the large vendors it used to comfortably

beat in this area.

Figure 12: Business benefits for the Performance Ma nagement peer group (n=420)


� The Performance Management peer group includes companies whose

products are predominantly used in planning projects. This peer group shows

above average business benefits results and is led by BOARD followed by

Oracle Essbase and IBM Cognos TM1.

� Jedox and Cubeware hold fourth and fifth places, also scoring excellent

results.

4.65

4.97

5.24

5.30

5.52

0 2 4 6

QlikTech

Cyberscience

Tableau

Phocas

Dimensional Insight

4.11

4.35

4.49

4.52

4.90

4.92

5.19

0 2 4 6

Infor

Evidanza

Cubeware

Jedox

IBM Cognos TM1

Oracle Essbase

BOARD


15

Figure 13: Business benefits for the Small and Medi um Project vendor peer group (n=1308)


� The SME Project vendor peer group includes products that are primarily used

for small and medium sized projects (<500 users). The sample of Yellowfin’s

customers in The BI Survey 12 reveals its clear focus on small companies

with less than 100 employees. Yellowfin appears to serve its customers well,

scoring exceptionally high business benefits, head and shoulders above the

competition.

� Dimensional Insight and Phocas follow in second and third places

respectively.

3.58

4.11

4.11

4.28

4.35

4.49

4.52

4.64

4.65

4.73

4.90

4.94

4.97

5.19

5.24

5.30

5.52

6.31

0 2 4 6 8

Decisyon

Infor

Pentaho

Microsoft SSRS

Evidanza

Cubeware

Jedox

Microsoft SSAS

QlikTech

Bissantz

IBM Cognos TM1

TARGIT

Cyberscience

BOARD

Tableau

Phocas

Dimensional Insight

Yellowfin


16

Figure 14: Business benefits for the Large Enterpri se vendor peer group (n=535)

The Large Enterprise Project peer group includes products that are primarily used for

large enterprise projects (>500 users). In the Large Enterprise projects peer group, a

leading group of four products scored excellent business benefits: Arcplan, Oracle

Essbase, MicroStrategy and Information Builders.

3.33

3.45

3.55

3.76

4.50

4.56

4.92

5.24

0 2 4 6

SAP BO WebI

Oracle OBIEE

SAP BW

IBM Cognos BI


MicroStrategy

Oracle Essbase

Arcplan

Next Steps

The BI Survey Analyzer Demo

The BI Survey Analyzer is an interactive online tool which enables you to perform your own custom analysis of the survey data.

Click here to register for an online demo and find out how effective The BI Survey Analyzer can be in helping you to understand the strengths and weaknesses of BI products and trends in the BI market.

Purchase

To purchase The BI Survey 12, visit our website or contact us at:

Email: [email protected]

Tel: +44 (0)20 8133 8987

http://www.bi-survey.com/index.php?id=229

http://www.bi-survey.com/

bi survey 12 results updated -...

Documents