wwv2015: sander klous_we are big data

We are Big DataThe future of the information society

prof. dr. Sander Klous

Big Data Ecosystems in Business and Society

University of Amsterdam

Big Data Analytics

KPMG Advisory

[email protected]

@sanderklous

http://nl.linkedin.com/in/sanderklous

1© 2014 KPMG Advisory N.V., registered with the trade register in the Netherlands under number 33263682, is a subsidiary of KPMG Europe LLP and a member firm of the KPMG network of

independent member firms affiliated with KPMG International Cooperative (‘KPMG International’), a Swiss entity. All rights reserved. Printed in the Netherlands. The KPMG name, logo and

‘cutting through complexity’ are registered trademarks or trademarks of KPMG International.

Extreme expectations

https://www.youtube.com/watch?v=2vXyx_qG6mQ

Big, bigger, biggest




010101101110011101100111000010011101101110011111010001

010110000101

H

t

W

Z0

300 MB/s




Nobel prize 2013 to Higgs and Englert

9 pages with authors

(3 authors @ KPMG)

The new information society




Shared value with Big Data

http://www.visaeurope.com/en/newsroom/news

/articles/2010/validsoft_fraud_solution.aspx




Torrents in the music industry,

Bitcoins in the financial sector




Jobless recovery in a developing information society

http://www.futuretech.ox.ac.uk/sites/futuretech.ox.ac.uk/files/The_Future_of_Employment_OMS_Working_Paper_1.pdf

Accountants: 95%

The dark side of Big Data




Big Brother is watching you




The Target case: life changing events

Andrew Pole had just started

working as a statistician for

Target in 2002…

http://www.nytimes.com/2012/02/19/maga

zine/shopping-habits.html




Lack of transparency: LG and TP-Link (Philips) example

http://doctorbeet.blogspot.nl/2013/11/lg-smart-tvs-logging-usb-filenames-and.html

http://www.cbpweb.nl/downloads_pb/pb_20130822-persoonsgegevens-smart-tv.pdf

Big Brother?

That’s us!

Fundamental questions about

Big Data




The profile of the data scientist

Apples and Pears

Suppose we have two jars with apples and pears.

Our prior knowledge is that:

■ Jar A contains 10 apples and 30 pears

■ Jar B contains 20 of each

Fred picks a jar, without further evidence there is a 50% chance this is jar A (or B).

Fred pulls out a pear. The new probability that Fred picked bowl A is 0.75 x 0.5 / ( 0.75 x 0.5 + 0.5 x 0.5 ) = 0.6

Jar A Jar B

Simpson’s paradoxAlthough in this simple example Bayesian statistics appears to be straight forward, many subtleties arise

in analysis. One of the more famous pitfalls is called Simpson’s paradox:

http://en.wikipedia.org/wiki/Simpson's_paradox

Question:

Do you have a higher chance of survival when falling overboard with or without a life jacket?

P(Hn|E) =P(E|Hn)P(Hn)

Sum1N (P(E|Hn))

Correlation or Causality?

http://en.wikipedia.org/wiki/Simpson's_paradox




Mechanism Design (Nobel prize 2007)

Sandy

Pentland

(MIT)




Rules and regulations in a do it yourself society




Big Data and Society, an ethical perspective

A Practical Guide to Big Data




Boundary conditions




KPMG Analytics & Visualization Environment

(KAVE)

The Overview

Horizontally Scalable

Open Source

Configurable

Modular

Secure

The Implementation

Remote (or local) hosting

Dedicated hardware

Virtualized system

Secure internal network




Sharing insights

Sensitive

Sensitive Data

Source

Private

Local

KAVEPII

Personally

Identifiable

Information

Sensitive

Sensitive Data

Source

PII

Personally

Identifiable

Information

Public Data

Source

Shared

Remote

KAVE

100 %

Coverage

Mobile

AppWebpage /

interface Two way flow of insights

or limited public data,

complete client control

One way flow of data

Trusted Third Party (A) Trusted Third Party (B)

Private

Remote

KAVE

Two countries / clients

/ departments need to

combine and share

insights securely

No single attractive target for hackers




Governance in a data driven organization

D&A Board

The D&A board is accountable for the success of D&A projects. The board consists of multiple delegates from the operational and D&A units. The board is headed by an executive that reports directly to the CEO.

From ideas to projects

The board is responsible for the vision, plan of approach and formation of project groups. Project groups consist of talent from within the organization to work out their own initiatives and ideas. Participation in these projects is by invitation only and considered a career accelerator. Project development occurs in short cycles.

Becoming ‘data driven’

As the D&A units mature, the division of data-oriented functionalities into operational silos decreases.

Understanding D&A services

The D&A units provide a variety of solutions to the operational units.Operational units will continue to run and be responsible for their own activities and associated data.

From data to ideas

The D&A services are managed by the board. Ideas are collected from all sources within the organization (not only from within the D&A Services groups).

Aggregating data

The D&A units facilitate the operational units to capitalize on data through data driven business models. D&A Board

Executive Committee

Bu

sin

ess

Mar

keti

ng

Fin

ance

HR

Operational units

Rea

l tim

e an

alys

is

Pro

cess

ing

Idea generation

VisionPlan of

approachProject Groups

D&A units

Dat

a A

cqu

isit

ion




Agile project management

Insight into

digitalizationDiscovering the digital possibilities

Breaking the

rules

Related worlds

Ambition

Ambition

Customer

perspective

Powertrack

planning

Business

value

Ideas for

digitalizationVision Generating an appetite

Preparation for and creating client cases using a

Big Data opportunity eventProof of Concept for selected cases and

hand over

Transformation of the solution

and business model

Envisioning…

uncovering client

cases using a Big

Data opportunities

Preparing…

selected client cases

for Proof of Concepts

Reviewing…

evaluating the results

and determining the

way forward

Developing…

the Proof of Concepts

of the selected client

cases

Coordinating the project using a phased and agile approach

Continuous interaction to orchestrate Big

Data strategy process and if necessary

adjust business case directions.

2 weeks 10 weeks to be decided

Part of this

proposal

Additional

activities

Client

cases

Applying Big DataStart Small




Behavioral patterns:

Dynamic online profiling

Conference venueBackground

Knowing your online web site visitors is advantageous since it enables

increasing the effectiveness of online offerings: by targeting more relevant

content, advertisements and e-commerce products to audiences with

interests that match, conversion rates and thus revenue will increase. Data-

driven insights based on web access logs help to create this understanding

in customer behavior.

Requirements / Challenges

■ Processing and combining large volumes of data from two separate ‘silos’

with little, but valuable, overlap: identified customer (subscription) data

versus unidentified customer (online) data

■ Automatized identification of meaningful clusters of interests in data

Resources

■ Data sources: web access logs and online subscription data

■ Duration (develop phase): 8 – 16 weeks

Solution

■ ‘KPMG Analytics & Visualization Environment (KAVE)’ cluster used as a

horizontally scalable storage and computing resource

■ Analysis algorithms and machine learning tools were developed to apply a

dynamic segmentation model and identify clusters of customers interests

Benefits

A pilot testing such ‘Decision Engine’ shows a >25% increase in conversion

rate (CTR) which corresponds to 0,3 – 1,7m euros of potential gain in online

advertisement revenue for this specific case.

Avg. CTR: 0.063

CTR SWAP: 0.079 (+25%)

User profile A

Read:

car.com

Searched: BMW

User profile B

Read:

olympicsSearched:

Nike





Crowd & Mobility Management at a large Dutch Telco


An infrastructure company wants to use a sensor network that covers the whole of the Netherlands. The network generates terabytes of data that can provide more insight in the transport flows in addition to their own transport data. Based on these combined data, the infrastructure company can make more efficient logistics planning and adequately respond to disturbances.


■ Linking external sensor network data to internal data sources

■ Recognizing trends in historical data & real-time monitoring of network data

■ Clear and simple overview of large volumes of complex data

Resources

■ Data sources: infrastructure and network data, sensor network data

■ Duration (develop phase): 6 –12 weeks

Solution

■ Combining different data sources into a data institute

■ Data storage by Teradata unified data architecture including Hadoopcluster

■ Distributed analysis using Aster & MapReduce algorithms

■ Storm implementation for real-time data processing

Benefits

■ Reduction of IT costs by outsourcing to specialized data institute

■ Cost savings by acting on trends and disruptions adequately

■ Enriched data as a product for other parties in the transport ecosystem





Shopping behavior at a large international furniture retailer


Tracking the customer behaviour is important information which can be gathered using several types of sources. Wi-Fi signals can for instance be used to detect shopping patterns from customers, but one can think of many different data sources and techniques, like security cameras, iBeacon or Bluetooth.


■ Real-time collection and analysis Wi-Fi device signals.

■ Analysing the customer behaviour and plotting patterns, graphs and trend lines.

■ Storing sales data, data from RFID chips or motion sensors for extra information about certain products.

■ Storing other relevant (e.g. Social Media) information.

Resources

■ Data sources: Wi-Fi device signals. Additionally: sales data, RFID data, social media data, camera footage, iBeacon.

■ Duration (develop phase): 8 – 16 weeks

Solution

The analysis of any of these separate data streams will provide vital information about the shopping behaviour of customers. Combining all the data creates even richer information.

Benefits

The goal is to define which variables affect a purchase and make changes accordingly. For example, a store might deploy more salespeople, alter displays or put out red blouses instead of blue, according to a story on Bloomberg.




Background

Monte Carlo simulation to analyze operational efficiency of Operation Room

planning at a large academic hospital. A new blue print for OR planning was

created and needed to be evaluated on performance, expected production

numbers and where issues might arise.


■ Handling poor data quality

■ Translate highly flexible human decision-making to rules in model

■ Client involvement (expert knowledge) was needed throughout the

process in order to create an expert system for planning

Resources

■ Data sources: blueprint, historical OR usage data


Solution

Built a Monte Carlo simulation model to evaluate the expected performance

of the blueprint in various scenario’s and to quantify these results.

Benefits

With our results, the client gained insight into the expected performance of

the new blueprint, including the expected performance for several

alternative scenario’s.

Asset management and planning at a University Medical Center

Realit

yS

imula

tion

2012 2014




Background

A Dutch retail bank started a project combining internal and external data

sources. The goal is to model risk for a collection of mortgages based on life-

changing events in order to reduce costs on financially unhealthy clients. By

combining public information on an online house-selling website with internal

bank data, a model has been built to describe and predict the length of

mortgages and to analyze the difference in house-selling price and the

mortgage.


■ More than 200,000 mortgages & over 250,000 homes for sale on

Funda.nl, while limited or no understanding of customers selling houses

■ Privacy challenge: sensitive personal information needs to be anonymized

Resources

■ Data sources: internal bank data, open data crawled from Funda.nl


Solution

■ An application crawls the Funda.nl pages on a daily basis

■ The results are loaded into a Teradata Aster Cluster and combined with

bank information on customers and products

■ Mortgages and Funda.nl results are linked based on postal code and

house number

Benefits

Daily insight into the housing market: prices, locations and time and

understanding customers that offer a home or apartment for sale

Mortgage Risk Portfolio Management at a large Dutch bank

Data Hub CustomerAnalytics

InstituteCompany A

Company B1

3

3

2

en

cry

ptio

n

en

cry

ptio

n

Blue line:

financially

healthy clients

Red line:

clients from

Fin. Health

Dep.




Background

In a short Big Data pilot, a bank wants to research the possibility to find

indicators that consumers (customers) will get into financial trouble before it

actually happens. This can be input for proactively helping customers that

need financial aids, enlarging customer trust, cutting costs on special needs

customers and decrease losses on defaults.


■ In order to get hands-on experience the bank wants to use Big Data

technologies (Teradata cluster)

Resources

■ Data sources: bank transactions data and customer data


Solution

■ Pattern recognition of income and expenditure

■ Modelling the customer dynamically, as a response-system

■ Describe the transaction behavior as a coupled second-order linear

differential equation

■ Monte Carlo simulation on the customer behavior in order to do a

prediction of financial health.

Benefits

Insights in typical spending behavior of financially healthy and unhealthy

customers

Financial health monitoring at a large Dutch bank

Customer

needs

special

attention

Month

Fra

ction o

f all

spendin

g

Conservative

spending

How often does a customer spend an amount under 1000 euros?

Customers heading for a financially

unhealthy situation

Triangles hold financially

sustainable behavior

Financially healthy customers

Conservativity

index

Conclusions and Reflections




Conclusions

■ Think big, start small, scale fast!

■ Start at the data. Value of data cannot be determined

upfront. Work iteratively toward value.

■ Privacy rules and regulations change when moving

from small to Big Data. Big Data really is different.

■ Allow the team to experiment and be creative.

Embrace a fail-fast distributed risk approach.

■ Support a possible use-case with a business case so

everybody is clear on what to expect as a result.

■ Big data is about ideas, skills but most of all: data.

Team up to get access to complementary sources.

The Golden Case:

1. Adds value for society and clients.

2. Shows the potential of Big Data

3. Starts small but is meaningful.

4. Has the potential to scale fast.




Maybe trust is overrated

© 2014 KPMG Advisory N.V., registered with the trade register in the

Netherlands under number 33263682, is a subsidiary of KPMG Europe LLP

and a member firm of the KPMG network of independent member firms

affiliated with KPMG International Cooperative (‘KPMG International’),

a Swiss entity. All rights reserved. Printed in the Netherlands.

The KPMG name, logo and ‘cutting through complexity’ are registered

trademarks or trademarks of KPMG International.

Sander Klous

KPMG Management Consulting

Laan van Langerhuize 1

1186 DS, Amstelveen

Tel: +31 20 656 7186

[email protected]

wwv2015: sander klous_we are big data

Marketing

kpmg advisory

subsidiary of kpmg europe

kpmg talkbook a4

registered trademarks

swiss entity

trade register

big datathe future

information societyprof