contact center metrics, contact center planning, and how our metrics can lead us down the wrong path

Ric KosibaPresident

Bay Bridge Decision Technologies

CONTACT CENTER METRICS, CONTACT CENTER PLANNING

(and How Our Choice of Metrics Make Us Do Silly Things)

© 2012 Bay Bridge Decision Technologies, Inc. All Rights Reserved. Proprietary and Confidential2

Your Seminar Leader

Ric Kosiba serves as President of Bay Bridge Decision Technologies. In early 2000, he cofounded the innovative software company and now leads the development of the company’s optimization technologies used in call center management. He is expert in the field of call center management and modeling, call center strategy development, and the optimization of large-scale operational processes. Kosiba received a Ph.D. in Operations Research and Engineering from Purdue University and an M.S.C.E. and B.S.C.E. from Purdue’s School of Civil Engineering. Kosiba has obtained a patent on the application of optimal collection strategies to delinquent portfolios in addition to two patents on the application of simulation and analytics to contact center planning.

At the start of his career, Kosiba served notable roles for two major airlines including Manager of Customer Service Analytics for USAir’s Operations Research Division as well as Operations Management Senior Analyst with Northwest Airlines. His specialties included airport and call center staffing as well as productivity improvement projects. Following this role, Kosiba moved into Customer Support at First USA, where he served as Vice President of Operations Research. Expertise here included all facets of contact center process improvement, ranging from overall collections strategy modeling to detailed staff plan development and call center budgeting. Prior to Bay Bridge, Kosiba held a position as the Director of Management Science at Partners First, where his primary duties included detailed modeling of portfolio risks, as well as predictive and prescriptive marketing and operations engineering.


Overview

This webinar will bounce around a lot! We’re going to chat about metrics and planning, and things that I’ve seen we do that don’t always make a lot of sense

Service failures and “catching up” Occupancy as efficiency (and what is better) Service level and back office Forecast error Staffing over/under


An old and common story…

… and spends the rest of the month trying to “catch up”

The call center gets hammered on the first day of the month…

Scenario:

• Service Level Goal: 85/20• On first two days, only

achieved 45%• From then on, overtime

by 5% service level in order to get average service level back up to 85% (run 90% every day)!


So, how does this work out?

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 1840

50

60

70

80

90

100

Service Level by Day

Daily Service Level

Avg SL for Month

Serv

iceL

evel

Achieve service level goal by day 18

Days


What does this cost the company in overtime?

About 22 FTE’s worth of overtime, for 16 days, at $20/hr, equals ~$57,000

22 FTE


But what would happen if our goal was, instead, an ASA goal?

Using CenterBridge’s sensitivity analysis, we can find “equivalent average speed of answers”

From CB Sensitivity Analysis:

• 45% SL = 80 Sec ASA• 85% SL = 20 Sec ASA• 90% SL = 10 Sec ASA


So, let’s do the same exact analyses, but in ASA, not SL

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 180

10

20

30

40

50

60

70

80

90

Average Speed of Answer by Day

Daily ASA

Avg ASA for Month

Aver

age

Spee

d of

Ans

wer Hit ASA goal by day 14 (12

days of playing catch up)

Days


There are strong diminishing service level returns

13%

10%

9%

5%3%

Buying service level is expensive at the upper ends of the curve

(it is hard and pricey to achieve a 90% SL)


It is also difficult and costly to maintain a 10 Second ASA

18 Sec

13 Sec

10 Sec

8 Sec3 Sec


Service Level!!

First. This is not a discussion of ASA versus SL

Each metric has it’s own properties– SL has a ceiling (100%)- and it is very

difficult to get near to that ceiling!– ASA’s floor is also impossible to achieve

• But it is easier to average to a number “20” when your floor is “0” and your performance is “10” than it is to average to an “85” when your cap is “100” and your performance is “90”

• But none of this has anything to do with “service”

ASA!!


The punch line

• You would save ~$12,000 or 22% of the overtime dollars, if you managed to an ASA goal instead of a service level goal, and you wanted to “catch up” to your service goal by month’s end

• But “catching up” is really pretty counterproductive– Nobody who called during the service blow up got better

service during the “catch up” days– Those who did call during those catch up days noticed nothing– It cost an awful lot of money to catch up – for no benefit

(except punitive)

• I realize that if you are an outsourcer or a utility, there are serious penalties for not hitting goals

• That does not mean it is a good idea service-wise


So what would be a better metric?

• ASA is slightly (22%) better in this scenario

• There is a WFM trend to manage service by “% intervals met”– The incentive then is to save costs by missing peaks and averaging poor service

peak intervals with high service valley intervals– CenterBridge weighs service by “minute” or “volume weighted”

• Heavy volume intervals have more impact• No “games” (and you are right staffed anyway)

The problem with service contracts– If you use outsourcers, does it make any sense to hold them to it? (don’t we

want our partners to succeed??)– Why would we do it to ourselves (Burn overtime hours and spend $57K with

little real benefit?)?– If an outsourcer, can we discuss the folly with our partners? Work on a better

contract?


Occupancy as Efficiency

Occupancy is at 88% this

month

They are waiting for a call to arrive We can cut your

budget by 12%

What? They are sitting around doing nothing for 12% of their time??

WFM

Finance


Occupancy measures economies of scale!

Lower volume,Occupancy at 70% SL = 62%

Higher volume,Occupancy at 70% SL = 72%


Occupancy does not measure true efficiency

Service Level is too high and occupancy low!

Let’s have a team meeting! Wow! You are

efficient again!

Look how inefficient your operation is!

WFM

Finance


So what are better measures of efficiency?

Via Michele Borboa and Duke Witte (and, hence, in CenterBridge!):

– Occupied to Staffed Time: Occupancy– Staffed to Worked Time: Of the time in the building, how much time agents

are available for contacts? (measures on premise “other stuff”)– Worked to Paid Time: Of the time being paid, how much time is being spent

in the building? (measures on premise to off premise efficiency)– Occupied to Worked Time: The ratio of time in the building to time on the

phone (measures on premise other stuff and economies of scale)– Occupied to Paid Time: Of the total paid hours, how much time is spent on

the phone?– Staffed to Paid Time: Of the total paid hours, how much time are agents are

available for contacts? (my bet: this is the best efficiency metric)


How do we as an industry determine how many agents we need?

Most: Erlang C

Some: Assumed Occupancy Workload Calculation

Fewer, but growing: Discrete-event simulation


What’s wrong with assuming occupancy first?(words of wisdom from Steve Martin)

If you know the right number of people, you know the occupancy. If you know occupancy, you know the right staff.

Guessing the occupancy is the same thing as guessing the right number of staff!

Steve Martin, Call Center Planning Savant

(Occupancy is a result of hiring, overtime, undertime, and controllable shrinkage decisions)


Erlang over-staffs all the time. Sometimes a lot. Sometimes not.

9 11 13 15 17 19 2160

80

100

120

140

160

180

200

Erlang vs Actual Staffing Requirements

actual

erlang

Hour of Day

Effec

tive

Staff

Req

uire

d

9 11 13 15 17 19 210

50

100

150

200

250

300

350

400

450

Erlang vs Actual Staffing Requirements

actual

erlang

Hour of Day

Effec

tive

Staff

Req

uire

d

Depending on workload calculations or Erlang will make your FTE requirements a guess


A properly validated model (this is discrete-event simulation)

Tip: Validation of your analytic process breeds confidence in both your analyses, and you! Make validation a regular part of your planning meetings– even if everyone is tired of reading how smart you are!


Service level to staff for back office

• Long service times (e.g. 80% responded within 24 hours)• Do our normal ways of calculating service levels make sense?

– Forecast each time period– Determine staffing independently

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6

Volume 1000 1000 1000 1000 1000 1000

Staff Required 200 200 200 200 200 200

Overflow Volume 0 200 400 600 800 1000

Volume (including overflows) 1000 1200 1400 1600 1800 2000

Example (80% / 24hrs):

Volumes grow and grow!


What about changing our normal way of staffing?

• Forecast each time period• Determine staffing knowing the overflow

Volumes reach close to a steady state!

Day 1 Day 2 Day 3 Day 4 Day 5 Day 6

Volume 1000 1000 1000 1000 1000 1000

Staff Required 200 240 245 245 245 245

Overflow Volume 0 200 240 248 249.6 249.9

Volume (including overflows) 1000 1200 1240 1248 1249.6 1249.9

We’ve spent a fair amount of time studying this: a better method is to staff to complete the work in a 24 hour period


Forecast everything (attrition, wage rate, each shrinkage category,…)!

An error rate of 5% of call volume is equal to an error rate of 3% of shrinkage!

Forecast Error

There is a relative value associated with each forecast’s error.

The value of each forecast’s accuracy is represented by the amount of service level error that the performance driver forecast produces (you can determine this using sensitivity analyses graphs)


Forecasting is one Piece of the Planning Life Cycle

THIS is the result of your forecast!

Error rates associated with the forecast is not nearly

as important as the errors associated with your plan!

“The end result is not a forecast, but a plan” -- Duke Witte, Wyndham Hotel Group


Lets Discuss Forecasting Error

• Its good to measure forecast error, but DO NOT GET HUNG UP ON IT

• Rule of thumb: always be suspicious when someone touts a forecast method based upon fancy error formulas. Statisticians are notorious for measuring the wrong things

• The real measure of forecast error is risk to the organization- either in service or cost

• Example: One method may have great goodness of fit, but be off more during peak periods- this error will overstaff significantly, when determining your hiring plan

• In order to measure forecast risk, you need an accurate staff/capacity planning method.

Ask yourself- which of these forecasting models will lead me

to a more reasonable

business Decision?


A quick example of forecast error: Which forecast is best?

Forecast #1 Versus Actual Volume

0

20,000

40,000

60,000

80,000

100,000

120,000

J J J F F M M A A M M J J J J J A A S S O O N N D D

Time

Cal

l Vo

lum

e

Actual Volume

Forecast #1


0

20,000

40,000

60,000

80,000

100,000

120,000

J F M A M J J A S O N DTime

Cal

l Vo

lum

e

Actual Volume

Forecast #2


0

20,000

40,000

60,000

80,000

100,000

120,000

J F M A M J J A S O N D

Time

Cal

l Vo

lum

e

Actual Volume

Forecast #3

Forecast Mean Error Mean Absolute

Error

RMSE Comments

Number 1 No Bias High Very High Variability and Low Confidence

By and Far The Worst Finish

Number 2 Small Under-forecast Bias

Low Low Variability and High Confidence

The Winner!

Number 3 Small Over-forecast Bias

Low Low Variabilityand High Confidence

A Close Second Place Finish!

What about Business Risk??


A quick example of forecast error: Which forecast is best?


0

20,000

40,000

60,000

80,000

100,000

120,000

J J J F F M M A A M M J J J J J A A S S O O N N D D

Time

Cal

l Vo

lum

e

Actual Volume

Forecast #1


0

20,000

40,000

60,000

80,000

100,000

120,000

J F M A M J J A S O N DTime

Cal

l Vo

lum

e

Actual Volume

Forecast #2


0

20,000

40,000

60,000

80,000

100,000

120,000

J F M A M J J A S O N D

Time

Cal

l Vo

lum

e

Actual Volume

Forecast #3

This method overstaffs at peak

This method understaffs at peakThis method staffs perfectly, just a week late

Assessing business risk:

Which forecasting technique would cause more harm to the company?


An over/under analysis

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49 510

100

200

300

400

500

600Expected Requirements Versus Staffed

Staffed AgentsExpected Require-ment

Week

Nu

mb

er

of

Ag

en

ts

The number of agents required by week

The number of agents staffed, using hiring, overtime, undertime, training, etc…

Over under is only half of the picture– the cost of hitting our goal. Is that the only decision we can make? (A: Nope.)

The other half of the picture is operational performance expected.

And the difference? Our over/under picture!


Service: ASA, SL, Abandon, Occupancy

Evaluating risk requires a (validated) simulation

With simulation, you can change anything and see resulting service (and vice versa). Accurately.


Final Thoughts

• Just because you have the power to do something, it doesn’t mean you should: Penalties for missing service should be used sparingly- why would you want your outsources to “catch up” and take a meaningless cost hit?

• Similarly, construct smart contracts: Our metrics and our contracts may create some counterproductive behaviors

• Challenge conventional wisdom: Metrics, such as occupancy and service level are easy to get, but may not measure what we think (i.e. efficiency)

• “How we’ve always done it” should be challenged when it comes to new contact types: Just because our methods worked for call centers does not mean that they will for contact centers

• Make sure you focus on the decision: Interim metrics like forecast error should not take priority over analyzing the best staffing decision given we know there will be variability


Supporting Tools: CenterBridge

CenterBridge is a contact center forecasting, strategic planning, and what-if analysis system. It helps you, for example:

– Forecast all center planning metrics– Quickly develop budget plans that are accurate and generate savings. Automatically

produce variance analysis– Perform risk and sensitivity analysis of your contact center– Set optimal service levels – Evaluate center investments, consolidation, and growth opportunities.

CenterBridge compliments tactical workforce management tools by focusing on strategic decision making

Uses a patent-pending, customized discrete-event simulation model of your contact center (not Erlang equations) to drive analysis


Contact Us!

Ric Kosiba

[email protected]

410-224-9883

… if you would like a copy of the slides or to see a quick CenterBridge demonstration

Also! We have a white paper, Contact Center Planning: Agility is Key, available for download at:

www.BayBridgeTech.com

mailto:[email protected]

http://www.baybridgetech.com/

contact center metrics, contact center planning, and how our metrics can lead us down the wrong path

Technology