Group 6

1

Your logo

Evaluation, Measurement & Refinement

Group # 06

1

Introduction

Evaluation Metrics, Needs & Necessity

Agenda

Discussion

Different Metrics

Usage and Limitations

Check PointsCheck Points

Summary & Conclusion

2

Group 6

2

KCRM Implementation Process

3

We Are Here

“Not everything that can be counted counts.and not everything that counts can be counted”

- William Bruce Cameron

Something to Keep in Mind

William Bruce Cameron

“That what can be measured is not always important, and what is important cannot always be measured.”

OR

4

and what is important cannot always be measured.- Amrit Tiwana:

Essential Guide to Knowledge Management.

Group 6

3

What is Business Metric?

A business metric is any type of measurement used to gauge some quantifiable component of a company's performanceperformance.

5

What is Business Metric?

Business metrics are part of the broad parea of business intelligence, which comprises a wide variety of applications and technologies for gathering, storing, analyzing, and providing access to data to help

t i k b tt b i

6

enterprise users make better business decisions.

Group 6

4

Metrics essentially define Measure of KCRM Success and serve as indicators of Business Impact on Customer Knowledge management

Why Business Metrics are Essential?

indicators of Business Impact on Customer Knowledge management.

Future Investments and Relationship Building depend so heavily on these indicators of success that poor choice of metrics can spell disaster.

It i t l ti l t I t ibl A t h

7

It is extremely essential to measure Intangible Assets such as Customer Loyalty, Relationship Value and Knowledge.

It is essential that Metrics focus on past as well as future events.

Requirements of a Good Evaluation Metrics

It is essential that metrics balance internal and external knowledge integration sufficiently well.

The most effective metrics that are needed the most are metrics that

8

can be measured today with impact on future outcomes.

Group 6

5

TCO

BenchmarkingTobin’s q

Matrices

KCRM Evaluation &

Measurement

Stages of Growth

Quality F ti

Financial ROI

Balanced Score Card

for E

valuati

Function Deployme

nt

9

on

Traditional Metrics

T t l C t f O hiReturn On Investment (ROI)

Tobin’s Q Total Cost of Ownership(TCO)

10

Group 6

6

Traditional Metrics Cont..

Return On Investment (ROI)

Time to Value

Dollar Value on Intellectual Assets

Cost Reduction (Dollars Saved)

Time to Payback

Prevailin

g A

pp

ro

Raw ROI

11

& Customer Relations

oach

esTobin’s Q – Firm’s market value and the total cost of replacing its physical assets.

Traditional Metrics

Total Cost of Ownership – TCO is the total life cycle cost associated with the purchase. The purchase includes implementation, maintenance & support of

the system. Measures beyond its initial cost of implementation.

12

Example : The cost of software licensing fee, H/W expenditure, training, integration, design and maintenance costs are all Included.

Group 6

7

Limitations of Traditional Metrics Although Companies don’t have trouble handling IT Investments, they falter on Soft Gains and Benefits. Decisions making becomes difficult where limited quantitative data exists. Soft Measures which are more difficult to measure or are not measured at all.

Soft Measures like; Customer & Employee Satisfaction &

13

p yLoyalty Customer and Employee Productivity Organization Stability and Leadership etc.

What is Benchmarking?

Benchmarking

When organizations want to improveWhen organizations want to improve their performance, they ‘benchmark’. i.e. they compare and measure their policies, practices, philosophies, and performance measures against those of high-performers within or outside the

Example: In baseball, you could argue that seven consecutive World Series Championships made the New York Yankees

14

g porganization, industry or anywhere in the world.

made the New York Yankees the benchmark.

Group 6

8

Four Benchmarking Steps

Select Benchmark Parameter – Select the customer interaction process, knowledge service processes and their scope for benchmarkingknowledge, service processes and their scope for benchmarking.

Select the Benchmarking Team – Ensure that it is sufficiently diverse from a functional mix point of view

Select Target Partners – Internal, from rival firms and from non rival firms or from averages representing your industry or sector.

More on

15

g g y y

Collect & Analyze Data- Record the benchmark data for future comparisons and use it to iteratively improve the benchmarked processes.

Benchmark Targets Advantages Disadvantages

Internal Business UnitsBreaks down Internal Communications Barriers & Easily

Internal Competition Might Impede UseCompared Internally Not Against

Benchmarking Targets

AccessibleCompared Internally Not Against Superior Industry Performers

Competing FirmsMeasured against Direct Competition, Partners Easily Identified

Legal Complexity“ESCROWS” Impose additional Costs

IndustryMeasured Against the Overall Market

ExpensivePrivacy Threats Might Surface

Cross IndustryInsights From Non Competing Firms

Comparative Population Not Truly Representative of Industry/ SectorHarder SellMore Expensive

16

Group 6

9

Provides readily usable methodology for measurement of process capability and technological knowledge

Bohn’s Stages of Knowledge Growth Framework

technological knowledge.

This relatively simple method gives a bird eye or a 30,000 foot view and clear bigger picture,

This framework, however, doesn’t give a low level view.

The following table, provides the references against which you can map,

17

The following table, provides the references against which you can map, evaluate and measure your business relative to your competitors and your industry, and causal associations and perceptive decisions.

Stages of Growth Framework

Stage Name Typical Form Of Knowledge

0 Complete Ignorance Does Not Exists Anywhere

1 Pure Art Pure Art

2 A areness Kno ledge is Primaril Tacit2 Awareness Knowledge is Primarily Tacit

3 Measure Knowledge Is Primarily Written

4 Control Of The MeanWritten and Embodies in Methodological routines

5 Process CapabilityLocal Recipes and Operating Manuals

6 Process Characterization Empirical Equations (Cost Reducing Trade offs are Known)6 Process Characterization Empirical Equations (Cost Reducing Trade-offs are Known)

7 Know Why Scientification - Procedures, Methodologies & Alogorithms

8 Complete Knowledge Knowledge Nirvana – never Happens

18

Group 6

10

Stages of Growth Framework

Stage Knowledge StageKnowledge

CharacteristicsLocation of Knowledge

Work ProcessLearningMethod

0Complete Ignorance

Cannot TellGood StateF B d

Undefined Undefined UndefinedFrom Bad

1 Pure Art Pure ArtTacit & In The Expert’s Head

Trial & Error Based

Look for emergent Patterns

List of PossiblyLargely Tacit, can be P ti ll E d

Experts DictatedP

Experts KeepRepeating P

2 AwarenessList of Possibly Relevant Variables Exists

Partially Expressed by words Gestures Or Diagrams

Process Execution That Is Still subject to Randomness

Processes While Looking for Emergent Patterns

19

Stages of Growth Framework

StageKnowledge

StageKnowledge

CharacteristicsLocation of Knowledge

Work ProcessLearningMethod

3 Measure Pretechnological

You Are Able to Decide Which Variables are more Important By

Casually Ambiguous Patterns Begin to

Repetition and Experimentatio3 Measure Pretechnological

p yNoting Their Correlation With The Desired Result.

Patterns Begin to Emerge

Experimentation

4Control of the

Mean

Scientific Method Feasible

Written & Embodied in Hardware/Software processes to some Extent

Some Knowledge can be Explicatedand codified. But cannot Measure qualitative Factors

Records of Processes And Their Outcomes Are Maintained

q

5Process Capability

Local Repeatable Recipe

A Semi-reliable Recipe based on Experience is developed and it may be codified.

Methodology Based Typically Produces Good Results.

Search of Satisfaction Patterns That produceSatisfactory Results

20

Group 6

11

Stages of Growth FrameworkStage

Knowledge Stage

Knowledge Characteristics

Location of Knowledge

Work ProcessLearningMethod

6Process

Cost-effective, Well-Developed,Recipe Capable of

Knowledge is documented in a

Mechanized, Automated, Time –proven, Repeatable

Continuous Application of Proven

6Characterization

Recipe, Capable of Handling Some Contingencies.

methodology that Usually Works

Methodology That Exhibits Localized Adaptability

Methodology will Surface its Weakness

7 Know Why

Formal or Informal Scientific Quantitative Model Is Developed

Relevant Knowledge is well Explicated and Codified; Contingencies can be handled.

Codified in Computer Software and process Manuals

Iterative Refinement

O i l

8Complete Knowledge

NirvanaRarely possible Any Contingency can be handled perfectly

KM becomes a NaturalPart of Work Process

Occasional Variation Resulting in The Inability to apply Processes Push it Back to Stage 7

21

Basic measurement method for KCRM effectiveness

T i i i t ti d i th t t ti

Quality Function Deployment

Turn vision into action during the strategy execution process.

Facilitate translation of high-level goals to discrete action and integrate inputs such as:

Individual goals

Perception of significance

22

Desired outcomes from various stakeholders

Into decomposable, measurable and more manageable actions

Group 6

12

Quality Function DevelopmentInterrelationship between

parameters Level of correlations

between the metrics and performance outcomes. denotes high, is

medium and is low.

Desired outcomes are

listed. Whether low- or high-level, these

outcomes must be clearly

observable

Relative priority is assigned to each outcome by attaching

weights. (Scale or percentage)

23

observable.

Speed Accuracy Cost Reliable

=5 high relationship=3 Medium relationship=1 Low relationship

1.Increase customer retention by 6% by the end of this fiscal year

2.Reduce problem resolution times by 35% in 18 weeks

3.Increase customer satisfaction levels with our

Speed Accuracy Cost Reliable

5

4

3

24

satisfaction levels with our service by 18% by June 2013

3

128kb/s 95%(2012) $50 90% (2012)

64kb/s 99% $45 95%

7 44 26 40

Customer-focus decisions

7=4*1+3*1

4 1 3 2

Group 6

13

The bank has been operating in the Slovak financial market

since 1993

Case: QFD application of a Slovak Bank

Is well recognized as a corporate and retail bank possessing

and advanced technology platform

Not promoting the quality of online services to a sufficient level

The QFD application was carried out in cooperation with the

25

The QFD application was carried out in cooperation with the

Electronic Banking and Development department.

Step 1: Identify customer requirements

Case: QFD application of a Slovak Bank

Data was collected in the form of customer survey and analyzed in several steps:

Iterative completion of a list of all the suggestions

Suggestions’ clustering according to their affinities

Definition of demanded quality criteria

26

Definition of demanded quality criteria

Clusters allocated among the demanded qualities

Prioritization of service quality criteria

Group 6

14

Step 2: Translation of the customer quality criteria into the service

Case: QFD application of a Slovak Bank

technical characteristics

Determining the service quality attributes

Evaluating the relative relationships among the customer quality criteria and the service quality attributes

Assessment of the implementation difficulty of the service quality attributes

27

Calculating technical significances for each service quality attribute

Competitive analysis based on the customer quality criteria and also on the service quality attributes

Group 6

15

Limitations of QFD

Can get over complex over time

The method does not distinguish between financial

and non-financial outcomes

29

It is regarded as one of the most comprehensive metrics

Balanced Scorecard

metrics

It is a measurement method that simultaenously checks and balances concerned units’ goals to achieve overall business vision

Here each card represents one dimension being tracked

Group 6

16

It balances

Long term & Short term objectives;

Balanced Scorecard

Financial & Nonfinancial measures;

Lagging & Leading indicators;

Internal & External Perspectives

It links strategy technology competitiveness customer centricity & Knowledge MgtIt links strategy, technology, competitiveness, customer centricity & Knowledge Mgt

But it requires extensive customization before application to business

Translates lofty vision into more doable, realistic, manageable and specific

performance driven goals

The Advantages

performance driven goals

Establishes direct linkages between financial and non-financial value drivers

Provides snapshot of the health of the firm at any point of time

Sufficient critical performance drivers monitor

Communicates KCRM strategy throughout the firm & links individual goals with

overall vision

Group 6

17

Financial Perspective

Goals Metrics Targets Initiatives

How It Works ?

The Customer Perspective

Goals Metrics Targets Initiatives

Internal Business Perspective

Goals Metrics Targets Initiatives

KCRM Vision

The Learning & Growth Perspective

Goals Metrics Targets Initiatives* Number ofperspectivescan be addedbut <7

Implementation Of BSC

Vision Translation Communication

li h k Feedback Reality Checks Feedback Incorporation

Group 6

18

Reaching managerial consensus on the key objectives behind the plan

Vision Translation

Decomposing vision into concrete goals

Create short term, specific goals for individual employees and relate them to organizational vision

Communicate vision-derivative goals and employee performance targets to the relevant departments & individual

Communication

pemployees

Clarify metric rewards linkages to demonstrate what is in it for them

Group 6

19

Evaluate how well selected metrics, individual level goals business unit

Reality Checks

individual level goals, business unit targets and allocated resources align with vision

Evaluate your goals, metrics and targets based feedback from customers

Feedback Incorporation

and employees

Incorporate this feedback to incrementally refine and refocus performance targets

Group 6

20

TheoryTo

PracticePractice

The Vision

Return on Capital Cash Flow

Project Profitability

Reliability of Performance

Example from Harvard Business Review

“As our customers’preferred provider,

we shall be the

industry leader.

This is our mission.”

Strategy

Services that Surpass Needs

Customer Satisfaction

Reliability of Performance

V Value for Money Competitive PriceHassle-Free Relationship High-Performance Professionals

Innovation

Shape Customer Requirement TVender Effectiveness Quality Service Safety/Loss Control

Continuous Improvement

Quality of Employees

Shareholder Expectations

Safety/Loss Control Superior Project Management

Continuous Improvement

Product and Service Innovation

Empowered Work Force

Group 6

21

Your logoCustomization to the Balance ScorecardOther employee and customer related measures worth considering:

Average expense per dollar earned

C t d l tLevel of customer attrition

Competence development expenses

per employee in dollars

Customer satisfaction

Delivery time deviation rate

Dollar figure value of employee

Marketing expense per customer

Number of ideas implemented from customer

suggestions

Renewal expense per existing customer

Time spent per unsuccessful sales leadattrition

Employee attrition rate

Employee satisfaction

Time spent per customer per year in dollars

Training expenses per employee

Analysis ROI Drivers Quantified Metrics

Shopping patterns Higher customer acquisition Increased revenue/sales

B2C E-commerce transaction analysis for the balanced scorecard

pp g p g q

Visitor behavior Higher customer retention Lower churn

Buyer: Browser ratio Churn reduction Higher acquisition rates

Cross-selling/Up-selling Effective personalization Higher sales/customer

Fulfillment Improved targeting Reduced operations costs

Product returns Finer market segmentation Improved profit margins

Advertising effectiveness Reduced fulfillment time

Business rule performance Increased value to customer

Customer profiling Higher stickiness

Segment-level targeting Advertising effectiveness

Product affinity

Demographics and geographies

Group 6

22

Analysis ROI Drivers Quantified Metrics

Root cause CSR throughput Lower support costs

Customer service benefits analysis for Balanced scorecard deployment

Root cause CSR throughput Lower support costs

High and low performers Schedule optimization Higher support volume

Process bottlenecks Resource allocation High MVC retention rates

Scheduling MVC retention and loyalty Price premiums

Up-sell/Cross-sell Higher service levels

Customer profiling/satisfaction Faster problem identification

E l ti /h tEscalations/hot cases

Call scheduling

Customer satisfaction

In 2004 about 57% of the global companies are working with the

Success stories of Balanced scorecard

In 2004, about 57% of the global companies are working with the balanced scorecard*

Used by various no. of government agencies, military units, business units and corporations as a whole, non-profit organizations, and schools

Examples include: Al Khamsa (an Architectural and EngineeringExamples include: Al Khamsa (an Architectural and Engineering consultancy based in the Middle East), United Nations, Philips

Group 6

23

Criticism of Balanced scorecard

Not based on any proven economic or financial theory, and therefore have no basis in the decision sciences

The process is entirely subjective and makes no provision to assess quantities (e.g., risk and economic value) in a way that is actuarially or economically well-founded

Balanced scorecard doesn’t provide a bottom line score or a unified view with clear recommendations: it is simply a list of metrics

P f i l d i ti li k d di tl t b l d dPerformance appraisal and incentives linked directly to balanced scorecard can lead to reduction in productivity

Reasons for failure of BS Implementation

Lack of visionary leadership and active leadership engagement

Introducing the balanced scorecard for reasons other than those related to better business performance managementp g

Lack of cross-functional organizational/ employee engagement in developing the balanced scorecard

Not clearly linking balanced scorecard indicators with strategic objectives

Not changing balanced scorecard measures when required

Using Balanced scorecard performance results to punish for underperformance

Treating the balanced scorecard effort as a project and not assigning adequate resourcesTreating the balanced scorecard effort as a project and not assigning adequate resources to the ongoing use of the balanced scorecard

Not communicating /sharing the balanced scorecard results widely across the organization

Group 6

24

Examples of Failure

Store24 (convenience store retailer in New England): eventually abandoned the balanced scorecard over a two year period.

Arran Ltd. (UK financial services sector): implemented in the mid 1990’s, initially the program was perceived to have been a success. However, as they sought to increase their use of Balanced Scorecard within the organization they began to encounter problems, and g y gultimately these have resulted in the firm moving away from using the Balanced Scorecard.

Developing a clear understanding of how strategy is operational.

Balanced scorecard: Final Words & Guidelines

Communicating and more importantly developing a competitive strategy across the organization.

Encouraging staff locally to develop initiatives to support your organization’s strategic direction.

Building an holistic view of the organization and importantly, an early warning system of when things are going right and wrong.

Developing a performance measuring system that spans all levels in the organization.

Group 6

25

Using too many metrics

Risky reward ties

10 common Pitfalls

Hard to control

Hard to focus on

Neglect the “soft stuff”

Focus on past performance

Measure the wrong things

P f i dPerfect metric syndrome

Obsess over narrow e-business metrics

Measure quantity but not quality

49

YOU point of view

How to Avoid Such Pitfalls

Clear recognition of company’s goals

Make sure KCRM initiative relates with the

business strategy

Gain senior management support

Incrementalize the project by choosing the

relevant metrics

50

Group 6

26

Measures reflect success

Traditional financial metrics are insufficient

B h ki i i i b d i i lf

Summary

Benchmarking is a starting metric but not an end in itself

Devise a Balanced Scorecard customized to your business

Stages of growth framework provides readily usable methodology for comparative measure with a limitation that it gives only the bird-view and not a low-level view

Quality function development turns vision into actiony p

51

Only traditional financial focus may fall short

Ch i i h i i i l f ll f KCRM i l i

Learning

Choosing right metrics are critical for overall success of KCRM implementation

Although different metrics are available, only those should be used that are best aligned with the business strategy, culture and of course budget

52

Group 6

27

Any Questions ?y

THANKS FOR YOUR ATTENTION