Creating Big Data Success with the Collaboration of Business and IT Teams

By Edward Chenard

Edward Chenard- Started big data at Best Buy, was working in big data at GE before it was called big data.- Set up one of the first hadoop clusters in Retail and the Midwest.- Won tax innovation credits for my work on big data- Tekne finalist for big data innovation- Set up big data, data science and data visualization teams- Managed teams as large as 300 with product portfolios of over $4B- I spend my time in cold places

edward@echenard.com

Twitter: Echenard

Slideshare: Echenard

Everyone is Jumping on to Big Data

The Reality of Big Data• As many as 3/4 of big data projects fail according to one Gartner study. • The third is that 39 percent of the failure of Big Data project is attributed to the

fact the data is siloed and there’s not a lot of cooperation in gaining access to that data. Now that is the oldest problem in the history of IT. - Infochimps

• 1. They focus on technology rather than business opportunities.• 2. They are unable to provide data access to subject matter experts.• 3. They fail to achieve enterprise adoption.Terradata's top three reasons why big data projects fail.

• Lack of alignment. Business and IT groups are not aligned on the business problem they need to solve but instead are tackling it from a technology perspective. Lack of true commitment from business stakeholders also makes alignment harder to achieve. Peter Sheldon - Forrester Analyst

Big Data at Most Companies

Big Data

IT Business

How a typical big data project takes place

• Someone hears about big data and then seeks funding.

• Other teams want to own it. Months of fighting takes place over ownership.

• The opportunity is either lost or the mission of the project gets altered.

• Teams work in silos, poor communication takes place as teams spend more time playing CYA. Achievement: Project failure with cost over runs, deadlines missed and lack of focus.

No One Team can Handle Big Data Alone

Current state of big data collaboration

Business(Strategy)

IT(Systems)

Analyst(Insights)

What does Big Data Really Mean to Business

“The ability to take data - to be able to understand it, to process it, to extract value from it, to visualize it, to communicate it's going to be a hugely important skill in the next decades, not only at the professional level but even at the educational level for

elementary school kids, for high school kids, for college kids. Because now we really do have essentially free and ubiquitous data. So the complimentary scarce factor is the ability to understand that data and extract value from it.”

Hal Varian

Everything is

about discovery

Why a focus on collaboration?

• Projects fail for simple reasons, lack of understanding the need for better collaboration and then knowing how to implement that collaboration, helps to ensure success.

• Failure does not need to be an option• Big data is the future of how we live and work,

but only if we get it right. Big data can be bigger than ecommerce in terms of impact on how we live.

Everyone Discovers

“The Data Discoverers looks a lot like you and me, but what’s different is their preoccupation with personal data.

They are relentlessly digital, they obsessively record everything about their personal lives, and they think that data is sexy. In fact, the bigger the data, the sexier it becomes.

Their lives - from a data perspective, at least - are perfectly groomed.”

data as a lifestyle

Data Discoverers

Data Discoverers are setting the trend in what will be common place in just a few short years.

More people will want to use their data and the consumerization of data and technology will continue.

As this trend goes, only organization that learn to merge the various disciplines of strategy, analytics and IT, will be successful

data as a lifestyle

Data Discoverers

How We Need to Look at Discovery

Horizon

Past

Discovery is the leading emerging interaction category of the Age of Insight

Complex ecosystems:

multi-channel experiences

Service models

everyware environments

Reactive datadynamic perspectives

Activity Centered Thinking

“Understand the quality performance of a system so I can better determine if I need to replace it.” - IT

“Understand a portfolio's exposures to assess portfolio-level investment mix.” - Strategy Manager

“I need to understand the customer trends in the data so I can better create models.” - Analyst

How Different Functions See the Same Issue

Identifying Modes

“Understand the quality performance of a system so I can better determine if I need to replace it.” - IT

“Understand a portfolio's exposures to assess portfolio-level investment mix.” - Strategy Manager

“Understand the customer trends in the data so I can better create models.” - Analyst

Mode = ‘Comprehend’ (understand)

Comprehending

‘To generate insight by understanding the nature or meaning of an item or data set’

e.g. “I need to analyze and understand consumer-customer-market trends to inform brand strategy & communications plan” – Director, Brand Image

Each Team has the same goal, to understand, what they may want to understand is often different but not exclusive or limit to the other team’s need to understand.

Identifying Modes

“I need visibility into the systems my colleagues are using in order to maximize the network ROI for the company.” - IT

“I need to identify customers/marketers/dealers failing & at risk of de-branding based on performance problems.” - Strategy

“I need to identify the best customer/consumer/region targets for our brand/products.”- Analyst

Mode = ‘Explore’

Modes are the verbs of discovery scenarios.

Locate

Verify

Monitor

Compare

Comprehend

Explore

Analyze

Evaluate

Synthesize

9 distinct modes

Where to Start

The Business Value Framework

Timeliness Automated and prompted

More Products Production Efficiency

Ease of Data Collection

Business value

Ease of Implementation

Customer Needs

Perceived Value

Pre-recorded Different

Products

Production Flexibility

Customer’s Wallet Share

Customer Acceptance

Value PerceivedEase of Data Collection Customer’s Wallet Share

Business value

Initiatives

Ease of Implementation

Customer Acceptance

Focus on Customers

Focus on Internals

How work gets structured

Vision & Goals

Governance

Execution

Clearly articulated vision for personalization and recommendations, precisely defined goals with how to measure. Defined scope of the product.

Market strategy, customer segmentation, prioritization, org focus, measurement and incentive systems

Production process, flexibility at scale, efficiency, relationship management, benchmarking, metrics, initiatives

Framing Collaboration

28

Big Data Collaboration

Value (Shared): Show me the money!?!

Analyst: Who, How, Where?

IT: What Tools and Why

Strategy: Where are you headed?

- Measurable Results- Multi-Channel Case Studies

- MapReduce, Hadoop- Cassandra, The Cloud- Pig, Hive,- HDFS

- Buy vs. Build- Open source options- Alignment with Analytical Infrastructure- Speed to Market- Privacy Considerations

- Data Scientist vs. Statistician- Where to find talent?- Retain, Train- Offshore vs. Onshore- University involvement

Always Remember: Data, Insights, Actions

Listen

•Listen to the data streams

Share

•Share the data with the rest of the organization

Engage

•Engage to the data to find the insights

Innovate

•Innovate new ideas from the insights gained from the data

Perform

•Perform insightful actions from the data to create better customer experiences

Collaboration helps to achieve where others fail.

Thank You!

• Edward Chenard – Twitter: Echenard– Email: edward@echenard.com– Blog: CrossChannelPrairie.com

Resources• Why Do Big IT Projects Fail So Often? (

http://www.informationweek.com/strategic-cio/executive-insights-and-innovation/why-do-big-it-projects-fail-so-often/d/d-id/1112087?)

• A statisticians view of big data (http://www.slideshare.net/kuonen/a-statisticians-view-on-big-data-and-data-science#!)

• Using Big Data to Create a Data Driven Organization (http://www.slideshare.net/echenard/using-big-to-create-a-data-drive-organization)

• The Language of Discover (http://www.slideshare.net/moJoe/designing-big-data-interactions-using-the-laguage-of-discovery?utm_source=slideshow&utm_medium=ssemail&utm_campaign=download_notification)

• Images by Emma Kim, Jason Maehl, David Spurdens, Evgenia Shadrina and Hill Street Studio