how to embrace big data - reply...2 big data in a nutshell 3 big data in italy 3 data volume is not...

16
How to embrace Big Data A methodology to look at the new technology

Upload: others

Post on 17-Jul-2020

2 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

How to embrace Big DataA methodology to look at the new technology

Page 2: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

Contents

2 Big Data in a nutshell

3 Big data in Italy

3 Data volume is not an issue

4 Italian firms embrace Big Data

4 Big Data strategies and operations need enhancements

5 The “Big” misunderstanding

5 How to approach Big Data effectively?

6 The Reply value offering

7 The technological perspective

7 Big Data as a ‘Washing Machine’

8 Traditional architecture as a data source for Big Data analytics

8 Traditional and Big Data architectures working together

9 Business perspective

9 Can Big Data help in detecting insurance fraud?

11 Big Data to improve ‘churn’ analysis in the telecoms industry

12 New boundaries in customer profiling

13 Conclusion

Page 3: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

How to embrace Big DataA methodology to look at the new technology

Page 4: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

2

How to embrace Big Data. A methodology to look at the new technology

Big Data in a nutshell

In it’s short life Big Data assumed a wide range of mean-

ings: on the one hand it refers to the global phenomenon

of information growth, resulting from the proliferation of

activities and data generated on the net via the social net-

work, smartphones or machine to machine interactions.

On the other hand, Big Data describes a new generation of

technologies and architectures, designed to extract value

in a cost-effective manner from very large volumes of in-

formation, by enabling high-speed data capture, discovery

and analysis.

In the already well-established technical literature, ‘three

Vs’ are generally used to characterize Big Data:

Volume: the total amount of data to be managed

Velocity: the pace at which the data can be processed

Variety: the complexity and heterogeneity

of the data set

Please forget it all! Big Data solutions cannot be defined

by how you can measure data in terms of Volume, Velocity and Variety. The three Vs are just measures of data related

issues. One firm’s “big data” is another firm’s peanut as

velocity appreciation strongly depends on any single con-

text behavior.

So, what is Big Data? A nice definition says aloud: “The

frontier of a firm’s ability to store, process, and access all

the data it needs to operate effectively, make decisions,

reduce risks and serve customers”: that’s probably the real

essence of the paradigm change addressed by Big Data

technology.

Over the last twenty years, ideas of how to assemble a

decision support system have coalesced around the con-

cept of a data warehouse as a tool for navigating business

issues but today the real challenge for business intelli-

gence is to let emerge hidden value through intelligent

filtering of low-density and high volumes of information,

being them operational or unstructured data arising from

sensors, transactions or either the web. Unfortunately the

unstructured data sources may not easily and cheaply fit

in traditional data warehouses, which may not be able to

handle the processing demand imposed by unstructured

data which for this reason remains largely untapped.

To help connecting the dots of all the content that’s out

there by analyzing a huge data set and returning results

in seconds a new class of technology has emerged; it in-

cludes new tools as NoSQL databases, Hadoop and Map

Reduce. These tools form the core of an open source soft-

ware framework that supports the processing of very large

data sets across clustered systems. Let we show you how

and why these technologies are gaining a leading position

in the interest of companies

Page 5: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

3

Big data in Italy

In January 2013, in collaboration with Forrester Consult-

ing, Reply carried out a survey to evaluate interest in adop-

tion of Big Data solutions in Italy. The aim was to explore

not just the acceptance of Big Data but also the stage of

maturity reached by organisations in building their strat-

egy towards Big Data implementation.

The study spanned vertical sectors across the country’s top

100 organisations, with concentrations in financial ser-

vices, telecoms, energy, utilities and waste management,

retail and professional services. Key findings included the

following, in such a way surprisingly, results:

Data volume is not an issue

The results of the survey confirm that a high percentage

of respondents do not have to deal with ‘petabytes’, ‘zet-

tabytes’ or ‘yottabytes’ of data. The most of the Italian

companies manage volumes of data that are relatively in-

significant in comparison with the vast size of major enter-

prises, being them social networks like Facebook, Twitter

or important retailers as Walmart and Target.

Anyway, Italian companies seem to have realised that vol-

ume is not the only or primary characteristic of Big Data.

The interest in Big Data technologies is then driven by

the necessity to acquire and process heterogeneous data,

while fastening computational time at a greater level of

accuracy. This is pretty similar to the findings of a recent

study conducted in the U.S., where it became evident that

the amount of useful data generated inside and outside

the company is not raising the hugeness of the major so-

cial network.

0 20 40 60 80 100

UNSTRUCTUREDDATA

ESTIMATE THE SIZE/VOLUME OF DATA WITHIN YOUR COMPANY

SEMI-STRUCTURED DATA

14% 24% 26% 25% 7% 2% 2%

STRUCTURED DATA FROM

TRANSACTIONALSYSTEMS

13% 15% 44% 18% 6% 3% 1%

11% 25% 24% 26% 7% 2% 3%

>1000TB 100-1000TB 10-100TB 1-10TB <1TB None Don’t know>1000TB 100-1000TB 10-100TB 1-10TB <1TB None Don’t know

Page 6: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

4

How to embrace Big Data. A methodology to look at the new technology

Italian firms embrace Big Data

Reply identified a significant amount of interest in Big

Data technologies and solutions. It looks as the wish to

take a competitive advantage from the analysis and inte-

gration of unstructured data is driving companies to adopt

Big Data technologies.

Notwithstanding only around a quarter of respondents

have already implemented a solution, 40% were planning

to implement in the next 12 months and a further 28%

planned stretching out to a slightly longer time horizon.

These companies are struggling with the data coming into

their organisations and are looking for new methods to

better leverage data to improve their businesses.

Big Data strategies and operations need enhancements

Key goals focus firstly on data quality, followed by busi-

ness objectives. The business cases that companies have

developed do not measure concrete key performance indi-

cators. Moreover, Italian organisations aren’t ‘pushing the

envelope’ when exploring the potential of Big Data.

Although the demonstrated significant interest for the new

technology, Italian organisations need definitely to invest

more attention in improving their Big Data strategies and

operations. When asked about the most important goals

or drivers organisation cares while overall orchestrating

its business intelligence strategy, 34% of respondents

pointed to improving data quality and consistency. But

data quality is not the end goal. The whole idea of Big

Data is to improve business success, through factors as

customer insights, operational efficiencies and cost con-

trol. Business targets must come first and data quality is

a prerequisite for these.

Only 11% of respondents claimed to have a business case

for Big Data with concrete KPIs and proven ROI. They

represent the highest level of maturity in the Big Data

initiatives. A further 19% reported having a business case

with KPIs but no proven ROI. The majority (47%) have a

business case with intangible benefits.

As shown by the results, 70% of respondents are not yet

able to translate the advantages of Big Data initiatives into

tangible business benefits. This would indicate that fur-

ther expertise is needed to lead Italian companies into the

Big Data world. Starting small and demonstrating tangible

benefits will enable organisations to prove the ROI on a

small scale before ‘going big’.

IMPLEMENTED,NOT EXPANDING

EXPANDING/UPGRADINGIMPLEMENTATION

PLANNING TO IMPLEMENTIN MORE THAN 1 YEAR

PLANNING TO IMPLEMENT INTHE NEXT 12 MONTHS

INTERESTED BUT NO PLANS

NOT INTERESTED

DON’T KNOW

“BASED ON FORRESTER’S DEFINITION OF BIG DATA, WHAT BEST DESCRIBES YOUR FIRM’S CURRENT USAGE/PLANS TO ADOPT BIG DATA TECHNOLOGIES AND SOLUTIONS?” (SELECT ONE)

3%

19%

28%

40%

10%

0%

0%

WE HAVE A BUSINESS CASEFOR BIG DATA WITH MEASURABLE

KPIS AND ALREADY PROVEN ROI

WE HAVE A BUSINESS CASEFOR BIG DATA WITH MEASURABLEKPIS AND A PROJECTED BUT NOT

YET PROVEN ROI

WE HAVE A BUSINESS CASEFOR BIG DATA BUT WITH

INTANGIBLE BENEFITS ONLY

CURRENTLY WE HAVE NOBUSINESS CASE, BUT WE ARECURRENTLY WORKING ON ONE

MAT

URI

TY

WE HAVE NO EXPLICITBUSINESS CASE FOR BIG DATA

DON’T KNOW

“DO YOU HAVE A BUSINESS CASE FOR YOUR BIG DATA INITIATIVE IN PLACE?"

19%

19%

11%

4%

0%

47%

Page 7: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

5

The “Big” misunderstanding

A cause of frustration for the customer trying to tap into

the ability to design and embrace a strategy about Big

Data is the fundamentally misleading view of Big Data as

a social phenomenon on the net, generated by millions or

even billions of pieces of information and backed by tech-

nologies that have been developed to extract the hidden

value of that data.

Too often technology key users (marketing or sales depart-

ment, product development team, security and fraud of-

fices, to mention just a few), are asking for solutions that

will never come because echoing the traditional approach.

It is not simply a matter of technology. Within the func-

tional organisation Big Data demands new processes, a

different way of interacting with the end customer, even

new skills to leverage the increased power of the analysis.

Simply Big Data requires a shift, in the corporate analyst

behaviors, to leverage the potentiality of new information

made available and in the IT departments, to deploy a

new array of IT architectures that will enable companies to

handle both the data storage requirements and the heavy

computational processes needed to handle cost-effective-

ly large volumes of data.

How to approach Big Data effectively?

The survey, in line with our overall understanding of com-

pany’s behaviours, suggests that the strategy to deal with

Big Data challenges will strongly differ depending on the

maturity of the organisation towards this topic. Reply has

developed a ‘Big Data maturity model’ to measure the or-

ganisation’s aptitude in approaching Big Data. The real

aim of this model is to help CxOs in better understanding

the company behavior alongside the new technology and

then properly identify the correct strategy for implement-

ing a coherent and profitable Big Data project.

We can segment company’s behavior into 4 blocks:

Inactive: Companies deal with Big Data issues as a

storage problem and essentially deny that there is a

problem. When issues come up, they just try to fix the

problem using standard techniques. This approach

results from a lack of business awareness and has

several failings:  it is expensive and unpredictable.

Proactive: Companies have the technologies and the

infrastructures to deal with Big Data but they still

don’t have business cases with measurable KPIs and

proven ROI.

Reactive: Companies have business cases and the

maturity to start a Big Data project but lack of ability

and expertise to address technological issues.

Active: Companies view Big Data as an asset and

own the necessary human resources, processes and

technologies to gain insight into their data.  These

companies looks at Big Data as an opportunity to dif-

ferentiate and gain competiveness, while well under-

stand it is not the last technological hype. The final

goal is then putting in place a comprehensive strategy

to maximize the data value to business purposes.

Page 8: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

6

How to embrace Big Data. A methodology to look at the new technology

The Reply value offering

As the model demonstrates there are several challenges on

both technical and organisational side that must be care-

fully addressed to achieve the full potential of Big Data,

while finding the right solution involves more than a simple

evaluation of price/performance of any specific tool.

Reply has built an integrated a consistent methodology

to support clients in the development of suitable strat-

egies to let them able to benefit of best of breed solu-

tions. A multidisciplinary team of business analysts and

technologists has been established to address the main

issues related to a Big Data project implementation within

a comprehensive approach. Additionally, to help business

people in challenging value from data, has been founded

a data scientists team. The goal is to help customers by

proposing the most appropriate business and technology

model fittings to their needs.

This heterogeneous team can help companies at any stage

of the Reply’s maturity model:

Inactive: The first stage, where the organisation has no

expertise in Big Data. Technologists provide the archi-

tectural solution, while business analysts and data

scientists collaborate with business in discovering new

patterns from the data, to create a business case with

a proven ROI.

Proactive: Organisation has already gained experience

in the technology but do not know how to apply it to a

real business case. Reply’s business analysts can help

supporting the development of a Big Data roadmap,

to transform customer’s needs into a real business

scenario. Data scientists work with business analysts

in finding new insight and perspectives, helping the

company to improve data value.

Reactive: Organisation has established a business

case with measurable KPIs but it lacks the technical

experience necessary to develop a Big Data archi-

tecture. Reply’s technologists can help customers in

finding the best architectural solution. The first step

is to analyse the organisation’s data sources and IT

infrastructure. The overall overview of the technical

environment enables technologists to develop a Big

Data infrastructure tailored to the customer’s goals.

Active: Organisation has here reached a high maturity

level in both the technology and business issues. Fine-

tuning job can still be done, however, to make it easier

to better develop business opportunities: this is the

typical environment where data scientists can, use their

expertise to refine the logical approach to data discove-

ry and modeling to deliver more detailed insights.

In summary, Big Data may be approached by two different

roadmaps: starting from the business issues, using Big

Data as a very powerful tool to redesign and improve data

analysis processes and from a technological perspective,

looking at Big Data solutions to reshuffle best practices

and infrastructure in order to provide faster and cheaper

results.

Page 9: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

7

The technological perspective

The most important value offered by the Reply’s approach

to Big Data implementation lies in the development and

delivery of solutions which strongly fits with the custom-

er’s technological architecture.

The goal is addressing the resolution of specific business

problems while maximizing the safeguard of the current

investments in technology, through a gradual integration

of the Big Data architecture into existing legacy systems.

Founding on the distinctive competencies and wide op-

erational expertise of the Group companies, Reply devel-

oped a framework to tailor Big Data implementation in

three different scenarios.

Big Data as a ‘Washing Machine’

A major problem when approaching a data warehouse solu-

tion is represented by extracting data from outside sources,

transforming and loading it into the data warehouse. ETL

processes (extract, transform, load) can involve consider-

able complexity while significant operational problems can

occur with improperly designed ETL systems, whereas – for

the business purposes – they represents a null-value, ex-

pensive and time consuming set of activities.

Big Data can solve this problem by substituting the tra-

ditional ETL process with a new kind of storage architec-

ture and, on top of this, a processing layer able to quickly

transform data and load it into a data warehouse.

This approach can appreciably lower the overall time

needed to satisfy the necessity of building the base of

reporting/analytics value chain, at a fraction of the cost

incurred by the traditional approach. It will also introduce

a faster management of the quality and coherence of the

data ‘ignited’ into the systems.

DATA

IN

GEST

ION

&

STO

RAGE

DATA

DATA

AN

ALYT

ICS

&

VISU

ALIZ

ATIO

N

PRESENTATION

CALCULATION ENGINE

DATA MART

DWH

STAGING AREA

CALCULATION ENGINE

DATA STORAGE

Unstructured Structured

Page 10: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

8

How to embrace Big Data. A methodology to look at the new technology

Traditional architecture as a data source for

Big Data analytics

Traditional data warehouses can handle many situations

but they do have limits. The volume of data imported into

a data warehouse is a critical issue in terms of costs to up-

grade the system and in data elaboration time. The higher

the volume, the greater the impact on processing perfor-

mance. The usual solution for this problem is to back up

the data but in most cases this is tantamount to losing

the information.

Big Data architectures can load data from existing data

warehouse systems and process it along with data from

sources such as data streams or unstructured data that

are not easily managed by the traditional data warehouse.

There are many benefits from using this approach; it is

possible, for example, to combine classical structured data

with other sources, enabling new insights and achieving a

better granularity in the data analysis. Furthermore, hav-

ing a Big Data storage structure means that data coming

from the data warehouse will never be lost; it will always

be possible to use historical data and analyse it.

Traditional and Big Data architectures working together

In some cases, thinking of replacing or supplementing IT

architectures can be valued as a disruptive approach, so

that the Companies prefer to keep their incumbent sys-

tems despite the loss of the information that - if properly

used - could dramatically upgrade their competiveness or

the revenue streams.

Big Data architecture, more than others solutions, allows

companies to implement a parallel infrastructure to ex-

ploit new data sources in counterpart with the traditional

B.I. ones. The cost-effective hardware jointly with the

open source software, which represents the foundations of

the Big Data solutions, enable a company to manage both

scenarios at a very marginal differential cost. Moreover,

some of the tools that belong to the Big Data ecosystem

(e.g. analytics, presentation and data integration layers)

can be substituted by or integrated with resources already

present in the traditional architectural stack.

DATA

AN

ALYT

ICS

&VI

SUAL

IZAT

ION

DATA

IN

GEST

ION

&

STO

RAGE

DATA

PRESENTATION

CALCULATION ENGINE

DATA STORAGE

DWH

STAGING AREA

ETL - ELT

Unstructured Structured

DATA

AN

ALYT

ICS

&

VISU

ALIZ

ATIO

N

DATA

M

ANAG

EMEN

T &

STO

RAGE

DATA

PRESENTATION

ANALYTICS

DWH

STAGING AREA

ETL - ELT

CALC. ENGINE

DATA STORAGE

Unstructured Structured

Page 11: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

9

Business perspective

Our daily confrontation with CxOs make clear that many

organisations start in claiming how business intelligence

solutions are failing to meet their current business needs;

this is the major push to accept looking at Big Data as

the ultimate instrument to design a new, more effective

information strategy. From being the sort of tool that was

only needed for meteorology or mathematical simulations,

Big Data has pretty recently moved into the industry main-

stream as the easiest and cheapest way to overcome tra-

ditional costs and implementation times of complex data

management systems, essential to encompass and manage

heterogeneous and multi source data sets.

Not all industries are likely to benefit from Big Data projects

equally and not surprisingly, the first movers were internet

companies; in fact, the most popular Big Data platforms

has been built on top of software originally used to batch

process data for search analysis but now retail, telecom,

financial services and media sectors are quickly recovering

while manufacturing and process industry are definitely ap-

proaching.

But just having the Big Data tools isn’t enough: enterprises

need to know what questions to ask, actually ask them and

then translate that into strategy or tactics. It will be impor-

tant for enterprises to develop new policies around privacy,

security and intellectual property. Big Data isn’t just about

technology and employees with the right skill sets, it will

also require businesses to align work flows, processes and

organization to get the most out of it. It is important to note

that enterprises should not concentrate on destructured

data at the expense of “current data” or business informa-

tion as normal. There is still a lot of value to be extracted

from the information inside their traditional databases!

Reply can help customers in designing and addressing the

right path to define an appropriate strategy, by identify-

ing business cases where a Big Data approach can create

a true difference to meet unsolved organisation’s needs.

Below are summarized some of the most common usage

patterns explored by Reply; while the explanation of the

usage patterns may be industry-specific, the rational basis

can be applied across industries to bring new sparks that

ignite the change.

Can Big Data help in detecting insurance fraud?

The technology that most insurers have currently in place

to help to fight frauds is a blend of business rules and

database searches, where the results rely heavily on the

sensitivity of the claims auditor. While these techniques

have proved being successful in detecting known fraud

patterns, insurers today need to invest in new analytical

capabilities to help them to spot unknown and complex

fraud activities. These analytical capabilities include in-

congruity detection, predictive modelling, unstructured

data mining and social network analysis.

Anomaly detection aims at discovering fraud by identify-

ing those elements that vary from the norm. Key perfor-

mance indicators associated with tasks or events are base-

lined and thresholds set. When a threshold for a particular

measure is exceeded, then the event is reported. Outliers

or anomalies could indicate a new or previously unknown

fraud pattern.

Page 12: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

10

How to embrace Big Data. A methodology to look at the new technology

Predictive models use past fraud events to produce fraud-

propensity scores. Adjusters simply enter data and claims

are automatically scored against the likelihood of them

being fraudulent. These scores are then made available

for review. Use of predictive modelling makes it possible

to understand new fraud trends.

Since around 80 percent of claims data is unstructured,

the use of tools able to mine unstructured data enables

insurers to analyse information arising from medical

chronicles, police records, external and internal database

sources or even e-mails.

Social network visualisation tools allow investigators to ac-

tually see network connections so they can uncover previ-

ously unknown relationships and conduct more effective

and efficient investigations.

By using Big Data technologies companies are able to

manage all of these issues and to ‘learn’ from experience

to improve their fraud detection and pattern identification

capability. This learning characteristic enables the soft-

ware to adapt and increase in sophistication as more and

more intelligence is gathered over time. The more analyti-

cal the tools, the higher the chance of detecting fraud in

the early stages and predicting potential areas of abuse

before fraudsters discover the opportunity themselves. Au-

tomation also means less reliance on the human element,

and provides greater accuracy and homogeneity in fraud

discovery activity.

Reply has established a proven methodology to apply a

Bayesian model in fraud recognition combined with Big

Data analysis techniques. This is a comprehensive ap-

proach, which includes data discovery through all the

available internal and external structured and unstruc-

tured data sources, combined with the powerful computa-

tional capabilities of a Big Data infrastructure to support

the claims manager in every phase of the investigation.

First of all, a network analysis will identify any histori-

cal relationship between the actors in a specific claim,

revealing any connection in the past that could suggest a

propensity to commit a fraud. Then a clusterization of the

actors and related behaviors based on a self-learning sta-

tistical model let emerge similarities in the data model, to

better represent relations and attitudes to plausible fraud

existence.

While this technology is still in its early stages, the bottom

line is that new Big Data analytics can be used to explore

large volumes of networked data, using high-speed pro-

cessing with configurable data entry from multiple internal

and external sources, to reveal fraudulent behaviour. Can

you imagine how far you could go using a so strong para-

digm change in tracking frauds?

Risk/TarifEvaluation

Internet data base

Externaldata baseClaims

ManagementsFraud

Monitoring

Real time evaluation

BRMS

Work�ow MgmtFraud reporting

SOGEI

MCTC

ANIA / ISVAP

Others

Case Analysis

Case Assignment

Case Manager Dashboard

Scoring

Data Certi�cationClustering

Network analysis

Big Data Analytics Data Matching

USERS ACTUARIALCLAIM MANAGER

RISKMANAGER

ContractsCustomersClaims

Fraudsblack list

Page 13: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

Big Data to improve ‘churn’ analysis in the telecoms industry

Today’s customers want competitive pricing, value for mon-

ey and, above all, a high quality service. They won’t hesitate

to switch providers if they don’t find what they’re looking

for. So particularly in mature markets or where regulations

and service dematerialisation makes ‘churn’ easier, it is ab-

solutely crucial to put in place a sustainable and robust

strategy for customer retention to preserve customer life-

time value. The telecoms market provides a good example

of why the high acquisition costs and slim profit margins for

each customer make churn analysis vital to help companies

identify and retain the most profitable among them.

In this context, the paradigm change ‘more is more’ is in

tune with the main aim of Big Data analytics. The uncov-

ering of hidden value, through the intelligent filtering of

low-density and high volumes of data, can become a real

differentiating factor. The more data you have, and the

more recent and accurate it is, the faster you can learn

from it and the more predictive you can be.

The value of Big Data can then be exploited in two dif-

ferent directions: to decrease the capital expenditure

(CAPEX) or operational expenditure (OPEX) associated

with the computational infrastructure needed to address

the huge amount of data used to feed predictive analyti-

cal models; and/or to increase the data sources used for

the integration and leverage of new kinds of unstructured

information, enabling companies to better describe and

understand customer behaviour.

One method now emerging to enable an operator to move

from reactive churn management to proactive customer

retention is to use predictive churn modelling based on so-

cial analytics to identify potential ‘churners’, thereby ena-

bling the operator to act on such predictions, rather than

waiting for explicit trigger points (e.g. credit on prepaid

card running down), by which time the churn is most prob-

ably inevitable, irrespective of any act or offer on the part

of the operator. Big Data analytics offer the opportunity

to process and correlate new data sources and types with

traditional ones, to achieve better results more efficiently

and receive insights that will set alarm bells ringing before

any damage has been done, so giving companies the op-

portunity to take preventive measures.

Pricing analytics and ‘next best offer’ recommendations

in particular are classic examples of how, by analysing

structured data (such as CDRs) and unstructured or semi-

structured data types (such as log files, IVR tracked calls

to call centres, clickstreams and, ultimately, text from

e-mails), telecoms operators can provide more accurate,

personalised offer recommendations.

Last but not least is the issue of timing. It is true that

traditional business intelligence solutions have allowed

enterprises to move forward by consolidating data sources

into centralised data centres. However, this data is used

‘simply’ for reporting. We are now moving into a new era

where information can and must be converted into real-

time actionable insight, to enable the company to respond

in real-time to behavioural changes in the customer mind-

set or to react quickly to threats on the competitive hori-

zon. This is exactly why and where Big Data analytics can

win the battle against ‘old’ BI tools.

VOICENETWORK

DATA

MOBILE WEBNAVIGATION

DATA

CUSTOMERINTERACTION

DATA

CELL TOWERSDATA

CRMTOOLS

AD SERVER

CAMPAIGNMNGT

CALL CENTER

HDFS &MAP

REDUCEREAL TIMEANALYTICS

BIG DATA PLATFORMOPERATIONAL

STACK

INSIGHTS

Feedback

11

Page 14: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

How to embrace Big Data. A methodology to look at the new technology

12

New boundaries in customer profiling

Customer analytics start with data. To get better customer

insight, most companies begin by analysing their struc-

tured transactional data, which typically includes infor-

mation such as demographics, purchase history, com-

plaints and retention information. Statistical algorithms

can help companies to create meaningful segments and

gain insight into buying patterns. These insights and ten-

dencies are then encapsulated in models which are used

as a basis for future predictions; basically, an extrapola-

tion of past history. Is this enough in today’s markets?

Probably not!

In recent years every one of us has become a powerful

‘walking data generator’, delivering personal information

(that reflects daily changes in our habits) through many

different channels. Information sources include call cen-

tre records, email communications and transactional data

as well as usage patterns on company websites. Very few

enterprises, however, are in a position to probe this ‘gold

mine’ of information.

In their quest to make these models more accurate, com-

panies are starting to embrace new sources of data; but

most of this data is unstructured and it is quite expensive

to have it integrated into traditional data warehouse and

data-mart infrastructures, both in terms of cost and time.

Moreover, analytical algorithms are continuing to evolve to

deal with the changing landscape brought about by new

trends (such as mobility, social media and e-Commerce),

while the need for a very fast computational time is in-

creasingly becoming a necessity to help companies to seg-

ment their customer base more effectively, attract more

profitable customers, improve campaign handling or re-

duce customer churn. Propensity models are also becom-

ing more dynamic to deal with the geo-spatial and tempo-

ral dimensions, acknowledging the fact that location and

time events impact people’s propensity to react to external

stimulation; in this case, the ability to react in real-time or

near real-time becomes a ‘must have’ feature.

As demonstrated by a recent Reply project, Big Data

technologies provide a very powerful tool-set to address

all of these issues. The ability to digest and elaborate in

real-time huge amounts of data as single cash lines in

till receipts, and compare them with the purchase his-

tory of each customer in order to generate promotions in

real-time is without any doubt a capability that would be

extremely hard to achieve using traditional analytics solu-

tions - which would in any event be prohibitively expen-

sive. The more data and information to be analysed, the

longer the process required (days); while Big Data solu-

tions allow retail companies to analyse huge volumes of

data, with more granularity, in a shorter period (hours vs.

days). Retailers can now get insight into customers’ sea-

sonal trends and use it to improve the management of

stock or create tailored pricing and promotions.

While embracing this new customer approach companies

must be aware there is a very fine line between using cus-

tomer analytics to create value by serving customers with

customised precision, and destroying value by surprising

customers with actions that erode trust. Privacy policies

and a consistent execution across the enterprise are es-

sential and must be properly performed to understand the

ever-narrower segmentation of customers and so deliver

much more precisely tailored products or services. It is

worth it, however, and the reward will surely overcome

best expectations.

Page 15: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

Conclusion

While other business metrics come and go, growth con-

tinues to be the most important criterion used to meas-

ure companies value, the measure by which the market

assesses companies and managers evaluate their perfor-

mance compared to competitors. Daily we appreciate as

competiveness passes more and more through a better

understanding of the huge amount of data organizations

collect and store about employees, customers, finances,

vendors, inventory, competitors and markets, to name only

a few. The amount of data needed is important because

people generally make better decisions if they have more

data available to them.

In parallel, even more in the coming years we will ap-

preciate the increasing volume and detail of information

captured by enterprises as the rise of multimedia, social

media and the Internet of Things will fuel exponential

growth in data for the foreseeable future. The real issue is

data have swept into every industry and business function

and are now an important factor of production, alongside

labor and capital.

As organizations will definitely understand this pattern

and invest to become more dependent on information,

the processes of gathering, managing, and utilizing data

will become more central to operational success, because

data is only as valuable as our ability to access and extract

meaning from it. This is probably the main reason why Big

Data solutions have definitely left their primordial field of

application, entering to its own right the industrial world.

Also if there could be reasons to be skeptical about the

Big Data expansion we can say without risk of contradic-

tion that a disciplined, targeted approach to Big Data de-

serves a very focused attention; when organizations will

recognize that Big Data’s ultimate value lies in generat-

ing higher quality insights looking in a different way to

available data to enable better decision making, interest

and related revenues will accelerate sharply. Albeit in this

field Big Data is still in its infancy, the rapid and constant

growth of attention to this technology suggests that indus-

try begin to embrace the challenge and is ready to take on

transformative measures, using the next generation of Big

Data industrial solutions.

Then, the final and most important question is: are you

ready to harness the power of Big Data?

13

Page 16: How to embrace Big Data - Reply...2 Big Data in a nutshell 3 Big data in Italy 3 Data volume is not an issue ... the necessity to acquire and process heterogeneous data, while fastening

www.reply.eu