big data industry insights 2015

This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its affiliates and is for the sole use of the intended Gartner audience or other intended recipients. This presentation may contain information that is confidential, proprietary or otherwise legally protected, and it may not be further copied, distributed or publicly displayed without the express written permission of Gartner, Inc. or its affiliates. © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Big Data Industry Insights

Lisa Kart @Kart_Lisa

http://denreymer.com

20% 10%

0%

30%

40%

50%

60%

Has your organization already invested in technology specifically designed to address the big data challenge?

Investments in Big Data Technology 100%

90%

80%

70%

2012 n=473

2013 n=720

2014 n=302

2015 n=437

Don't know No plans Plan within 2yrs Plan within 1yr Yes

Big Data Investments Continue to Rise but Slowing Down

64% 73% Percentage investing or

planning 58% 76%

3 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Key Issues


1. What are the vertical industry trends around big data?

2.  What business problems are top priority in different industries?

3.  Where should I focus?


55% 55% 53% 48% 47% 44% 42% 41% 39% 38%

26%

17% 15% 16%

14% 12% 14% 8%

23% 28% 23%

14%

7% 9% 13%

14%

6% 17%

21%

18%

33%

8%

21%

21% 19% 19% 24%

29% 17%

29% 18%

15% 33%

1% 1% 6% 8%

15% 5%

Don’t know

No plans at this time

Plan to within 2 yrs

Plan to within 1 yr

Have invested

Has your organization already invested in technology specifically designed to address the big data challenge? Total sample

Big Data investment – industry

n=

5

Percentage investing or

planning

Percentage investing

Retail/ Svcs Insur- Trans- Health- Banking Edu Manu & Utilities Comm./ Gov

Trade ance portation care N.Res. Media

29 78 32 21 17 59 24 80 18 13 57


Which of the following best describes your organization's stage of big data adoption?

100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0%

n=437

n=720

n=302

Despite the Opportunities, Organizations Struggle with Getting to Deployment

2013

2014

2015

Deployed

Piloting and experimenting

Developing strategy

Knowledge gathering

No plans to invest at this time

Don’t know

5% 31% 19% 18% 20% 8%

4%

24% 13% 19% 27% 13%

4%

21% 13% 18% 30% 14%


Which of the following best describes your organization’s stage of big data adoption?

State of Big Data adoption- by industry

6%

Services n=78 1%

Manufacturing n=80 1%

29%

21%

19% 1%

12% 18% 6% Percentage piloting or deployed-

Total sample 44%

14%

15%

31%

32%

36%

33%

19%

29%

21%

18%

10%

10%

7%

Healthcare n=17

Retail/Trade n=29

Education n=24

Utilities n=18

Insurance n=32

Comm./Media n=13

Banking n=59

Government n=57

Don't know if currently investing Currently investing/planning- don't know adoption stage Developing strategy Deployed

5% 14%

33% 12%

18% 18% 21%

17% 2% 15% 12%

52% 5% Transportation n=21

No plans to invest Knowledge gathering Piloting and experimenting

7

Percentage full deployment- Total sample 14%

15% 15% 0% 31% 23% 15%

19% 6% 9% 47% 16%

22 % 39% 22% 17%

29% 17% 17% 21% 17%


Rank 1 Rank 2 Rank 3

What are your organization’s top 3 hurdles or challenges with big data?

The Top Big Data Challenge Remains the Same

Sum

Determining how to get value from big data 33% 13% 9% 55%

Obtaining skills and capabilities needed 6% 16% 13% 36%

Risk and governance issues (security, privacy, data quality) 11% 11% 11% 33%

Funding for big data-related initiatives 9% 12% 10% 31%

Defining our strategy 7% 11% 13% 31%

Integrating multiple data sources 6% 11% 8% 26%

Integrating big data technology with existing infrastructure 3% 8% 13% 25%

Infrastructure and/or architecture 5%

7%

10% 22%

Leadership or organizational issues 6% 4% 8% 18%

Understanding what is "Big Data" 10% 3%2% 15%

Other 3% 3%3% 9%


Big Data Challenges Are More Practical As You Adopt

31%

20%

20%

10%

17%

36% 37%

42%

16%

21%

62%

14%

22%

20%

25%

20%

33%

30%

43%

54%

9%

14%

23%

30%

34%

23%

23%

44%

39%

53%

Understanding what is "Big Data"

Leadership or organizational issues

Infrastructure and/or architecture

Integrating big data technology with existing infrastructure

Integrating multiple data sources

Funding for big data-related initiatives

Defining our strategy

Risk and governance issues (security, privacy, data quality)

Obtaining skills and capabilities needed

Determining how to get value from big data

Have invested (n=192)

Planning (n=138)

No plans (n=86)

9

67%

47%

26%

24%

23%

22%

18%

17%

12%

20%

33%

42%

39%

36%

40%

45%

41%

34%

87%

80%

68%

63%

59%

62%

64%

58%

46%

Transactions

Log data

Geospatial/location data

Social media profile data

Emails/documents

Social media chat/interaction data

Sensor/machine-generated data (Internet of Things) Free-form text

Images

Currently Analyze (n=195)

Not analyzing today but plan to analyze in the future (n=138)


Which types of big data does your organization currently analyze and which do you plan to add to your analytics in the future?

SUM

Types of Big Data Analyzed- now vs. planned

Multiple responses allowed

Audio 9% 32% 41%

Video 8% 33% 41%

Other 10% 6% 16%

Key Issues


1. What are the vertical industry trends around big data? 2. What business problems are top priority in different industries?

3. Where should I focus?

Heatmap of Big Data Business Problems by Industry



Manu & N. Res.

Media/ Comm

Svcs Gov. Edu Retail Banking Insur- ance

Health- care

Trans- portation

Utilities

Enhanced customer experience

52% 78% 66% 43% 76% 83% 77% 77% 73% 69% 44%

Process efficiency 45% 33% 35% 49% 65% 43% 41% 50% 73% 69% 78%

More targeted marketing 43% 89% 53% 17% 41% 78% 66% 58% - 38% 17%

Cost reduction 42% 33% 35% 37% 35% 30% 41% 31% 45% 56% 61%

Improved risk management 14% 22% 29% 29% 35% 22% 52% 58% 55% 31% 61%

New products 23% 67% 37% 14% 24% 35% 27% 50% - 19% 33%

Developing information products

26% 33% 44% 31% 12% 22% 23% 19% 9% 19% 11%

Enhanced security capabilities 17% 22% 21% 34% 29% 13% 27% 27% 9% 19% 28%

Regulatory compliance 11% 22% 18% 23% 18% 9% 25% 23% 27% 31% 44%

n= 65 9 62 35 17 23 44 26 11 16 18

Tackling the problem of game scheduling

§  Opportunity –  Schedule NFL games to maximize profit

§  Data and Analytics –  20,000 variables and 50,000 constraints were analyzed using

‘FICO Xpress Optimization’ suite to come up with optimized schedule while evaluating 7000 game options

–  Best games schedules were selected which can fetch higher TV ratings and revenue opportunities

§  Results –  NFL’s revenue and sponsorship grew substantially after using

the solution in the last five years

–  Saves on time as new schedule can be produced in 24 hours, a task which could take months earlier


§  Opportunity –  A rapidly growing Turkish credit card business targeting lower value

segments caused an increase in fraud §  Data and Analytics

–  Replaced manual process of credit card application review with automated real-time scoring and flagging

–  Increase from 13% to 100% of applications reviewed –  Implemented fraud modeling in 15 days using KXEN

§  Results –  Increased number of identified actual fraudulent applications by 3x;

92% of fraud cases identified –  Reduced number of fraud alerts from 300,000 to 30,000 per quarter

by tuning and discovering new patterns –  Saving $25,000 per day; ROI achieved in one week

Finding Fraud Faster


§  Opportunity –  Improve citizen safety and save city resources

§  Data and Analytics –  The New York City Fire Department algorithm analyzes 2400

factors from 330,000 commercial and public buildings

–  Determines a risk score that guides inspectors to prioritize certain buildings and their likely fire safety issues

§  Results (TBD) –  70% success in identifying fire hazards in buildings

–  Reduce fires and other safety related events; Save on personnel and firefighting resources; Reduce insurance claims

Heat Mapping Potential Fire Risk Hotspots


§  Opportunity –  Offer credit to the underbanked – those without a credit history

§  Data and Analytics –  A sophisticated self-learning scoring model –  Up to 15,000 dynamic data points for each individual, including

social networks, mobile usage, location, e-commerce data etc.

§  Results –  Ability to lend to the 73% of those people with no traditional

credit scores –  <7% loss rate in established markets (lower than using

traditional credit scores alone) –  Loan payments in 15min vs 1-3 days

Giving Credit When Credit is Due


§  Opportunity –  Improve health and well-being of dogs

§  Data and Analytics –  Whistle: Wearable device and mobile app that tracks behavior of

your dog and compares it to baselines of similar breeds & ages.

–  FitBark: Simply, FitBit for dogs. Its "BarkScore" tells you how much activity Rover has had each day.

§  Results –  Alerts owners to possible health issues before they become

evident

–  How much is your dog walking, playing, resting?

–  Is your dog walker doing her job?

Big Data and Analytics Have Gone to the Dogs


§  Opportunity –  Increase the value, breadth, quality and consistency of

company valuation information available to subscribers of the Credit Suisse Holt Platform

§  Data and Analytics –  Generates and updates natural language narratives in real-time

to ensure they reflect company’s latest performance using Narrative Science’s natural language generation platform

–  Applies proprietary investment-research methodology to describe the operational quality, valuation, and riskiness of companies using data, risk, valuation and operations data

§  Results –  Increased company research available to subscribers by 250%;

eliminated issues with quality and consistency of language

Improving Quality of Financial Research, Naturally


§  Opportunity –  Understand activities within a building, without having to go

inside

§  Data and Analytics –  Indoor Atlas maps the interior of a building using the unique

magnetic “fingerprint” of the structure caused by distortions of the Earth’s magnetic field.

–  Determines location of internal structures and people to within six feet, including which floor

§  Results –  Emergency personnel can use a Google Map overlay to navigate

a building –  Competitors can see changes to layouts and foot traffic –  Pay $99/mo to keep your building’s magnetic map private

Buildings Have Magnetic Personalities Too


§  Opportunity –  Lack of consistent pricing, long negotiations and slow quote

turnaround for deals

§  Data and Analytics –  PROS pricing solution integrated millions of historical transactions

along with attributes to create pricing segments and derive pricing recommendations

–  Enhanced pricing envelope for over 500K specific price points based on the product, customer and deal type available on demand for new quotes

§  Results –  Customer-specific pricing based model led to 2% increase in profit margin –  Reduced quote turnaround time by 50% enabling auto-approvals,

leaving pricing analysts more time to spend on newer customers and strategic deals

An Offer They Cannot Refuse


§  Opportunity –  Improve reliability of the electric grid and the utilization of energy to

meet state renewables goals

§  Data and Analytics –  Real-time visualization and analysis of 25,000 miles of power lines –  Space-Time Insight, OSIsoft, Oracle –  Hourly reforecasting of generation needs based on wind and solar

estimates; real-time alerts for crisis conditions

§  Results –  No system-wide outages since implementation –  Enabled implementation of 4,000 pricing nodes (up from 5) to

facilitate cost-effective local market pricing –  50% improvement in renewable forecast accuracy

Keeping the Lights on in California


§  Opportunity –  Improve anti-money laundering investigation speed, cost

and transparency

§  Data and Analytics –  Automated data access, investigation and analytics via Pneuron in

single integrated “fabric”. Data from internal and external sources are presented as one complete package without replacing or rewriting existing hardware and software investments

–  Avoid need for data centralization by pushing the analytics to the data & employing non-invasive late-binding dynamic scoring model

§  Results –  Increased speed and reduced cost of investigation by 60% –  All AML alerts are consistent, auditable and documented, and

thresholds now can be set to zero

A Fresh, Clean Approach to Optimized Anti-Money Laundering Performance

Multinatio nal Financial

Ins t i tution


§  Opportunity –  Create defensible revenue forecast models to pass Comprehensive Capital

Analysis and Review (CCAR) “stress test”

§  Data and Analytics –  Correlated and analyzed 2600 macro-economic variables with revenue

streams for dozens of business units using Ayasdi’s machine intelligence software

–  Uncovered variable permutations that were hard to identify using incumbent analytics approaches and shortened the variable selection process – from three months to two weeks

§  Results –  Achieved the cleanest Federal Reserve test pass of top US banks –  Citi stock added $9B in market capitalization the following day; and

announced a dividend increase of 500%

Financial Stress Tests: Stressed No More


13%

4%

17%

35%

34%

38%

18%

Don’t know

Other

ROI is/will be measured by the non-financial impact to the company.

ROI is/will be measured by improvements to decision-making or process efficiency.

ROI is/will be measured by improvements to organizational effectiveness.

ROI is/will be measured by financial returns

ROI is not being measured/We do not plan to measure ROI

41% of Organizations Don't Know if Big Data ROI Will Be Positive or Negative


Positive ROI 57%

Negative ROI, 3%

Don't know, 41%

ROI Measurement


Positive/negative ROI

How is ROI being measured/How will ROI be measured for your organization’s big data investment?

Please indicate whether your organization's big data investment has /is expected to have - a positive or negative ROI?


How is ROI being measured/How will ROI be measured for your organization’s big data investment?

ROI measurement- by industry

Retail/Trade n=23

Comm./Media n=9

Insurance n=26

Services n=62

Manufacturing n=65

Utilities n=18

Government n=35

Banking n=44

Transportation n=16

Education n=17

Healthcare n=11

ROI is not being measured/We do not plan to measure ROI ROI is/will be measured

25

Please indicate whether your organization's big data investment has /is expected to have - a positive or negative ROI?

36%

40%

47%

51%

54%

61%

61%

63%

66%

70%

89%

3%

2%

4%

6%

2%

6%

2%

9%

64%

53%

48%

42%

33%

36%

31%

32%

11% Comm./Media n=9

Retail/Trade n=23

Services n=62

Transportation n=16

Banking n=44

Utilities n=18

Insurance n=26

Manufacturing n=65

Education n=17

Government n=35

Healthcare n=11

Positive ROI Negative ROI Don't know

9% 87%

22% 78%

12% 77%

15% 73%

18% 72%

22% 67%

14% 66%

20% 64%

38% 63%

18% 59%

36% 36%

Key Issues


1.  What are the vertical industry trends around big data?

2.  What business problems are top priority in different industries?

3. Where should I focus?

It's All About Analytics Maturity

Hybrid, integrated

Unstructured, external

Structured, internal, siloed

Ad hoc, batch,

offline analytics

Pervasive, real-time, embedded analytics

Dat

a

Increase analytics maturity by: •  Analyzing new data sources •  Broadening your portfolio of

analytic capabilities •  Applying analytics to

more decisions, faster

Descriptive Diagnostic

Predictive Prescriptive


Build Your Portfolio of Analytics Capabilities

Human Input

Data Decision

Predictive What will happen?

Diagnostic Why did it happen?

Descriptive What happened?

Prescriptive What should I do?

Analysis

Action

Decision Automation


Decision Support

Future

Past

Apply Relevant Data and Analytics to Decision Making

Create Awareness; a Decision Must Be

Made

Understand the Scope

and Context of the

Decision

Identify Likely

Outcomes

Identify the Best Course

of Action

Report on the Results

of the Action

Descriptive

Predictive Prescriptive

Diagnostic


Three Core Skill Areas Are Needed

Analytics Skills

IT Skills

Business Skills

Ask Good Questions

Decision Making

Transparent Versus "Black Box"

Which Analytics to Choose?

Data Exploration

High- Performance Computing

Build, Buy, Outsource

Feature Engineering

Know the Constraints (E.g., Legal, Ethics, Market)

Data Governance

Deployment


8% 5% 5% 6%

8% 10% 10% 10% 10% 11% 11% 12%

15% 16%

31% 32%

3% 5%

5%

5%

6%

7% 11

%

8%

5% 8%

13%

15%

17%

16%

25%

37%

2015 survey (n=333) 2014 survey (n=218)


Please indicate who initiated your organization’s big data initiatives

Both IT and Business Should Drive Big Data Initiatives


Art of the Possible

Seek Big Ideas Beyond Your Borders

I waited to see what leaders

in our industry were doing with data

I came up with some great ideas for using data on

my own

I worked with business

partners to develop new

ways to use data

We adopted and adapted winning ideas from other

industries for using data


Key Takeaways


üBig data is maturing, moving away from the challenges of data volume and variety, toward getting value.

üIncrease analytics maturity by: -  Integrating and analyzing new relevant data sources.

-  Moving beyond basic BI to diagnostic, predictive and prescriptive analytics.

-  More closely tying insights to business decisions.

üBuild your strategy around use cases, business goals and outcomes. The decisions will guide the required data and analytics.

Recommended Gartner Research


èSurvey Analysis: Practical Challenges Mount as Big Data Moves to Mainstream Nick Heudecker, Lisa Kart (G00289494)

èSurvey Analysis: Big Data Investment Grows but Deployments Remain Scarce in 2014 Nick Heudecker, Lisa Kart (G00263798)

èSurvey Analysis: Big Data Adoption in 2013 Shows Substance Behind the Hype Lisa Kart, Nick Heudecker, Frank Buytendijk (G00255160)

è Toolkit: Big Data Business Opportunities From Over 100 Use Cases Frank Buytendijk and others (G00252112)

è Extend Your Portfolio of Analytics Capabilities Lisa Kart and others (G00254653)

For more information, stop by Gartner Research Zone.


Questions?


Appendix: Methodology & Respondent Profile


What Is "Big Data"?

"Big data" is:

•  high-volume, high- velocity and high- variety information assets

•  that demand cost- effective, innovative forms of information processing

•  for enhanced insight and decision making.

VOLUME

VELOCITY

VARIETY

Methodology


38

§  This research was conducted via online survey in June 1-16, 2015 among Gartner Research Circle Members – a Gartner-managed panel comprised of IT and business leaders.

§  In total, 437 members participated by indicating their organization’s investment plans around technology to support big data: DEFINITION PRESENTENTED: Gartner defines “big data” as high volume, velocity and variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision-making. Ø  333 have invested, or plan to invest within the next two years Ø  91 do not have plans to invest at this time Ø  13 did not know

§  The survey was developed collaboratively by a team of Gartner analysts covering ITL Enterprise Software, and was reviewed, tested and administered by Gartner’s Research Data Analytics team.

NOTE: The results of this study are representative of the respondent base and not necessarily the market as a whole.

18%

18%

14%

13%

7%

5%

5%

5%

4%

4%

2%

1%

1%

2%

Manufacturing & Natural Resources

Services

Banking

Government

Insurance

Education

Retail

Transportation

Utilities

Healthcare Providers

Communications

Media

Wholesale Trade

Other

Respondent profile – company characteristics

n=437

Primaryindustry: AnnualRevenue:DK, 12% Non profit,

11%

#employeesworldwide:

DK, 0.00686498

9 <1000, 23%

<$500M, 25%

$500M - $1B - $3B $1B, 6%

, 13%

$3B - $10B, 13%

$10B +, 19%

n=437

1000-9999, 38%


39

10000 +, 39%

n=437 MEAN=9016

MEAN=$4101M

Respondent profile – region

14%APAC

42%EMEA

40%N.America

5%LaHnAmerica


40

n=437

Unknown 1%

Technology adoption profile

Aggressive 16%


41

Mainstream 63%

Conservative 21%

n=437

6%

43%

51%

Primarily business- focused

Blend of business and IT

Primarily IT-focused


42

Respondent profile – job function IT roles:

Business roles:

n=437

Enterprise Architecture 42% IT Infrastructure and Operations 36% Applications: Development, Integration, CRM, ERP, SCM and Portals, Content and Collaboration

30%

Business Intelligence and Information Management 30% C-level IT Executive Leadership 28% Security, Risk & Governance 27% Sourcing and Vendor Relationships 25% Business Process Improvement 24% Program and Portfolio Management 24% Other 5%

Business Strategy 53%

Business Unit/Executive Leadership 36%

Product Development and Management 21%

Marketing 11%

Sales 8%

Other 17%