creating a metrics-driven...

© 2009 Cisco Systems, Inc. All rights reserved. 1

For The Data Warehousing InstituteMonday, February 23, 2009

David Hsiao, Director, Corporate Quality Metrics & Benchmarking

Naznin Shroff, Senior Manager, Global Supply Chain Management

Creating a Metrics-Driven Organization:Measure the Work and Work the Measures


Metrics Driven OrganizationKey Elements

Culture Top to Bottom

Focus on Customer Success

No Boundaries; Cross-Functional

Data to Business Intelligence

Partnership & Framework

Internal Performance Measures

Governance

Customer Success


Customer SuccessCommitment from the Top

John ChambersChairman and CEO, Cisco

Quality is doing what you said you would do, when you said you would do it.

Quality is never letting the customer down.

Quality is being accountable for the total customer experience.


Customer SuccessFoundation of Cisco Culture


Customer SuccessKey Investments

Systemic Repeatable Process Improvement,

Organizational Learning

Cross-Functional Quality Initiatives

Customer Expectations, Compliance, Metrics

and Benchmarks

Essential Quality Information

Customer Intimacy, Satisfaction Survey,

Advisory Boards

Listen to Our Customers

Delivering an Unrivaled Customer Experience


Advisory Boards

Technical Assistance Center

Cisco Account Teams

Customer Survey

Executive Briefings

Customer Visits

Customer SuccessCustomer Listening Strategies


Customer SuccessMeasurement Taxonomy

Valued Partnership

Customer focused

Technically innovative

Easy to do business with

Attitudes and

Perception

Experience

•Reliable products•Usability

•Performance

•Predictability

•Easy to deploy•Ease of Upgrade

•Timely problem

resolution

•Responsive

Account & Channel

Teams


Align to Bellwether CustomersWorldwide Averages Hide Warning Signs

Qualify DeployRelease OperatePlan Develop Test


Align to Bellwether CustomersCustomer-Specific Views Reveal Variance

Qualify DeployRelease OperatePlan Develop Test


OverallSatisfaction

(nn%)

Increased Purchases

Price

(nn%)

Customer

Focused Image

(nn%)

Technologically

Advanced

Image (nn%)

Helps Business Succeed Image

(nn%)

Value(nn%)

Recommendations

Overall Products(nn%)

Hardware nn%Software nn%

Time to Deploy %

Upgrade Satisfaction %Reliability %

Technical Innovation %

Feature/Function %

Ease of Use %

Lead Time %Serviceability %

Performance %

Cisco Post-Sales Support %

Interoperability %

Technical Documentation % Acct Team Ability to Match to

Your Prob/Obj %

Network Security %

Product Training %

Partner Ability to Match to Your Prob/Obj %Partner Post-Sales Support %

Reliability %

Time to Deploy %

Technical Innovation %

Performance %

Lead Time %

Upgrade Satisfaction %

Feature/Function %

Interoperability %

Acct Team Ability to Match to

Your Prob/Obj %

Ease of Use %

Cisco Post-Sales Support %

Network Security %

Serviceability %

Technical Documentation %

Product Training %

Partner Ability to Match to Your Prob/Obj %

Partner Post-Sales Support %

Cisco Account Team(nn%)

Responsiveness %

Commitment to Your Business

Success %

Matching Prod/Svcs to Your Prob/Obj %

SE Skill Level %

SE Availability %

Communication – New Prod/Tech %

Cisco Post Sales Tech

Support (nn%)

Pre-Sales Channel Partner(nn%)

Cisco Order Management

(nn%)

Post-Sales Channel Partner(nn%)

Timely ProblemResolution %

POC Problem Resolution %

Acct Team Taking Care of Issues %

Professional Services %

Professionalism %Ability to Diagnose Problems %

Problem Escalation/

Management %

Telephone Support %

SLA’s Offered %Pricing Offered %

Timely Problem

Resolution %

Telephone Support %

TAC’s Ability to

Diagnose Problems %Spare Part Delivery/

Configuration %

Problem Escalation/

Management %

Effectiveness of TAC Website %

Advanced Services %

Trusted Advisor %Responsiveness %

Professionalism %

Matching Prod/Svcs to

Your Prob/Obj %

SE Skill Level %Acct Manager

Knowledge %

Impact on Business

Development %

Communication – NewProd/Tech %

Pricing Offered %

Reliable/ Dependable

Products Image(nn%)

Truly Loyal

Att

itud

e

High Risk Trapped

Accessible

Behavior

+

Š+

-

Truly Loyal

Att

itud

e

High Risk Trapped

Accessible

Behavior

+

Š+

-

Truly Loyal

Att

itud

e

High Risk Trapped

Accessible

Behavior

+

Š+

-

Truly Loyal

Att

itud

e

High Risk Trapped

Accessible

Behavior

+

Š+

Truly Loyal

Att

itud

e

High Risk Trapped

Accessible

Behavior

+

Š+

-

Loyalty

CSRM Satisfaction %

GCC Satisfaction %Ease of Placing Orders %

Meeting Promised Delivery

Date %

Promised Delivery Date

Meeting Request Date %Communication of Promised

Delivery Date %

People %

Time/Effort to Order

Prod/Svcs %Aligns w/ Business Needs %

Ease of Selecting

Prod/Svcs %

Ease of Post-Order Mgmt %

Customer SuccessDrivers of Loyalty


Customer Advocacy

MetricsTeam Lead

Hardware Quality Metrics

Team Lead

Customer Experience

Metrics Team Lead

Software

Operations

Metrics

Team Lead

Software Quality Metrics

Team Lead

QDI Sponsors

• Hardware Field Metrics

• Mfg Process

• Case Mgmt

• Mfg Quality Information Portal

• Customer Experience Dashboard

• Customer Reliability Performance Metrics

• Voice of the Customer

• QDI Usage &

Performance Tracking

• Root Cause /

Preventative Fixes

• OBIEE / Siebel

Analytics

• Data Model

• Case Mgmt

• IT Support

IT Architecture, Tools &

Operational Excellence

Team Lead

• Exec Quality

Information

Portal

• Quality

Integrated

Hierarchy

• Software

Ad-hoc

Reporting

• Software

Quality

Metrics &

Operational

Dashboards

• Software

Products /

Components

Metrics

Mgr, Quality Metrics

Mgr, Software Business

Operations

Mgr, Quality Systems,

GSCM

Mgr, ITQDI Architect

Mgr, Software Quality

Mgr, Engineering

Dir, EngineeringDir, Software Operations

Dir, IT SupportDir, Software

QualityDir, Engineering

Quality

Dir, QualityData

Infrastructure

VP, Corporate Quality

VP, Sales Strat. & Planning

VP, Gbl Supply Chain Mgmt

VP, IT SupportVP, EngineeringVP, Software Quality

Software Business

Intelligence and Tools

Team Lead

QDI Steering CommitteeChair

QDI Core TeamCore Team Lead

QDI Sub-Teams

Quality Data Infrastructure (QDI) Collaboration- Operating Structure

• Customer Satisfaction Dashboard

• Technical Assistance Center Dashboard

• Customer Assurance Program

Mgr, CustomerAssurance

Dir, Technical Support

VP, Technical Support

• Software Bug

Database and

Dashboard

• Software Release

Health Dashboard

• Engineering Business

Intelligence

• Software Contracts

Management

Reporting

Business IT & Tools


From Data to Business Intelligence


Topology of the Enterprise BI Infrastructure

Source Systems

Common DimensionsData Model - Single

Source of Truth Tier

Data Extract & Business

Rules Engine

Product

Customer

Time

Supplier

Organization

Roles

CMGeography

Others

…

•Customer Sat. & Loyalty

•Bookings & Shipments

•Install Base

•Service Contracts

•TAC assistance calls

•Service Requests

•Product Returns

•Hardware Defects

• Software Defects

•Others…

Incremental Development

3.0

2.0

1.1

1.0


Data Model- Single

Source of Truth Tier

Data Extract & Business

Rules Engine

3.0

2.0

1.1

1.0

Incremental Development

Topology of the Enterprise BI Infrastructure

Internal and External Users

Scorecards and Predictive Capabilities

Dashboards and Standardized Reports

Highly Iterative Model

Analytics Framework

Sales and

Marketing

Suppliers

Customer

AdvocacyEngineering

Customer

Service

Corporate

Quality

Global Supply

Chain

Management

Research &

Development


The Partnership & Framework


Business and IT Partnership model

•Strong partnership and commitment between the BI Steering committee and IT

•Co-development of the BI System development lifecycle methodology

•Well defined internal SLAs and metrics for Delivery, Operations excellence and Performance efficiency

•Strong Business and IT alignment with external vendors for ongoing feature set development


Organizational Change Management

Identify stakeholders, requirements,

assess business readiness

Design the solution

Validate the solution

functions as required

Verify solution meets the customer’s

needs, exercise change control

Release solution to

general availability, training and

support

Building Pervasiveness into the Framework:Organization Change Management Methodology

Define Requirements

Design Validate DeploymentPilot/UAT

Fail to plan, plan to fail

Carl W. Buechner

Success is complete

stakeholder adoption

Adoption


Building the Framework:Data Quality and Audits


Building the Framework: Metadata management and Data Governance


Building the Framework: Common Navigation


Building Agility into the Framework

Holistic visibility and alignment with Business models, transformations

Incubate new concepts via prototyping

Iterate around UI and Dashboard development

Evolve to a Managed BI environment leveraging best of SSOT and UI flexibility

Image taken from:http://www.bbc.co.uk/cumbria/content/image_galleries/barrow_aircraft_carrier25072007_gallery.shtml?2 Crown Copyright/MOD 2008

http://www.bbc.co.uk/cumbria/content/image_galleries/barrow_aircraft_carrier25072007_gallery.shtml?2


The Development Life Cycle: Sandbox and Enterprise environmentsEnterprise Gates

Concept Build Concept Ratification

Core Team

Gate BC CC

Proto/Pilot

Review and CC

for Enterprise

deployment

EC DC RR GO LIVE

Approval fromSteering

Committee

Core Team

(QCT)

Core Team

(QCT)Core Team (QCT) SME Team SME Team

Steering, Core,

Portfolio, SME team

UAT review

Architectural alignment/Prototype

Initiation where applicable

OCM Review

QDS Technical Architecture team

Core Team (QCT) - Inform

Core Team

Steering Comm.

Core Team and Technical Teams

Core Team

Core Team

Image Source: www.agilemodeling.com

Concept Implementation

Dashboards and Standardized

Reports

Highly Iterative Model

Governance and Change Management

Scorecards and Predictive

Capabilities

http://www.agilemodeling.com/


Measure Field PerformanceThe Customer Experience Dashboard

Sample Data Only

Iterative Dashboard Development methodology

Innovative model for iterative dashboard development by business teams


Sample Chart Managed BI Solutions


Sample Chart

www.agilemodeling.com

Managed BI Solutions

http://www.agilemodeling.com/


Measuring the Framework:Internal Performance Measures


Platform Adoption: Unique users


Platform Adoption: Number of Hits


Internal Client Satisfaction Scores

Operations Excellence


Accomplishments and Industry Accolades

Awarded Industry Best in Class in the Enterprise BI category

BI Framework designed for agility linked to Performance Management

Single point compliance enablement for Sarbanes-Oxley, ISO and TL9000

Unified Platform for external partners to drive continues improvements

Pervasiveness and Adoption:~3500 users, ~70,000 hits per quarter

Internal Customer Satisfaction scores:

–Project delivery: 4.2 out of 5

–Operations Excellence: 4.85 out of 5


Leveraging Business Intelligence for Customer Success


House of QualityIdentifies Improvement Impacts

CustomerExp Metrics

Cust Found DefectsDOA

RMA / MTBFHA / Outage

Service RequestOn Time Delivery

Defect DensityRelease Quality

SW DPM

Quality Initiatives

Failure AnalysisQuality Plans

DesignPhase Containment

System TestingRelease MgmtDiagnostics

CompensationCommunication

Customer Needs

SW QualityHW Quality

ArchitectureEase of Use

CommunicationPartners

Order FulfillmentSupportService

Acct Teams can better manage expectations with objective

Customer Experience measures

Executive Sponsors can

evaluate effectiveness of

initiatives portfolio

Infrastructure Teams can

improve measures of

Customer Experience


Governance and PrioritizationOrganic Collaboration Model

Line of Business

SEGMENTED GO TO MARKET

ENT

COMSP

MarketingChannels IBSG

Services / Support

Manufacturing

Sales

Finance / Legal / HR

BusinessDevelopment / Alliances

Command / Control

Command and Control

Collaboration and Teamwork

COLLABORATION and TEAMWORK

IBSG SalesMarketing

Channels

Manufacturing

Finance /Legal / HR

Business Development / Alliances

Services/Support

2001

2005

2009

CDO

CDO


Governance & PrioritizationHierarchy of Roles

• Long-term corporate strategy

• Allocation of corporate resources

• Accountable to CEO

• Issues affecting councils or functions

• Alignment of cross-functional teams

• Accountable to councils and/or functions

• X-functional issues that materially affect Cisco; e.g. business strategies, big bets, company productivity

• Champions investments and/or priorities

• Alignment of cross-functional resources

• Accountable to Operating Committee

• Specific near-term issues

• Execution or targeted initiatives

• Accountable to boards, functions, or councils

Lo

ng

evit

y

OC

Councils

Boards

Working Teams,e.g., Task Forces,

Tiger Teams, PMOs etc.


Customer SuccessLinks Directly to Compensation

CompanyPerformance

Individual Customer

• Revenue Targets

• Profitability

• OPEX / CAPEX

• Annual Customer Satisfaction Survey

• All Customers

• All Markets

• All Technologies

• MBO’s

• Market Results

• Team Results

• Teamwork & Collaboration

Cisco Bonus =

Applies to all staff


Customer SuccessHistory of Customer Satisfaction

Source: Cisco annual survey; measured on a scale of 1 to 5 in fiscal years

1995 20081996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007

5

3

3.984.01 4.06

4.154.22

4.33 4.334.30 4.33

4.37 4.37 4.364.41 4.44

10+ years of history—formally tracking customer satisfaction

Central part of Cisco’s culture

Customer satisfaction tied to the bonus plan


Metrics-Driven OrganizationKey Take Away Messages

• Listen to Customers• Align to Bellwether Customers• Link Internal Process Measures ->

Customer Experience -> Revenue • Leverage Scalable, Actionable

Single Sources of Truth• Establish Cross-Functional

Governance for Initiatives• Reinforce Culture through

Rewards & Recognition

creating a metrics-driven...

Documents