building a quality assurance program for 2-1-1 centers using ‘voice of the customer’ research

©The Fraser Group,2014

BUILDING A QUALITY ASSURANCE

PROGRAM FOR 2-1-1 CENTERS USING ‘VOICE OF THE CUSTOMER’

RESEARCH BARRY MANERS, MANAGING DIRECTOR, THE FRASER GROUP

LISA AUSTIN, DIRECTOR OF 2-1-1 STRATEGIC ENHANCEMENTS AND DISASTER RECOVERY UNITED WAY WORLDWIDE


United Way Worldwide


The Fraser GroupVoice of the Customer Solutions

RA Road AmericaA MAPFRE Company

amadeus

National 311 Performance Benchmark

http://www.farmbureauinsurance-mi.com/index.htm

http://www.nlrb.gov/

http://www.agccp.org/


The Agenda

The 211 “mystery caller” benchmark program Consensus reality, aka “Why does quality

assurance matter” Lessons learned from the 211 “mystery

caller” benchmark program The path forward


The Agenda

The 211 “mystery caller” benchmark program

Consensus reality, aka “Why does quality assurance matter”

Lessons learned from the 211 “mystery caller” benchmark program

The path forward


The Agenda

The 211 “mystery caller” benchmark program

Pilot creation


How the 211 “Mystery Caller” was created

Consultation Determined Scope of Work Worked with 211 US Steering Committee Utilized AIRS Standards Fraser Group Best Practices Florida Involved 211’s


Participating 2-1-1 Centers

1 - Broward County 2-1-1 2 - NJ Partnership 2-1-1 3 – United Way of Escambia County 2-1-1 4 – United Way of Northeast Louisiana 2-1-1 5 – WIN 2-1-1 6 – United Way of Greater Atlanta 2-1-1 7 – United Way of Greater Cleveland 2-1-1 8 - 2-1-1 LA County 9 – United Way Association of South Carolina 10 - NJ Partnership 2-1-1 (second location) 11 – 2-1-1 Palm Beach 12 – Michigan 2-1-1 (Gryphon Place)


The Agenda

The 211 “mystery caller” benchmark program Consensus reality, aka “Why does

quality assurance matter” Lessons learned from the 211 “mystery caller”

benchmark program The path forward

10

Referral/Loyalty

CallerPerceived Value

OverallQuality ofProducts &

Services

Cost of Acquisition & Usage

Market Presence & Reputation

The Customer

Events PromotionsThe Contact

CenterSupport Marketing

Assuring the caller has the best experience. Does it matter?

11

The Filter

Strategy

Goals/Objectives

Mission

Vision

Referral/Loyalty

Caller Perceived Value

OverallQuality of

Products &Services

Cost of Acquisition

& Usage


The Caller

The Organization

The Contact Center

The Contact Center

Understanding the filter.

12

The Hawthorne Effect

Strategy

Goals/Objectives

Mission

Vision

Referral/Loyalty


OverallQuality of

Products &Services

Cost of Acquisition

& Usage


The Caller

The Organization

The Contact Center

The Contact Center


13

Frederick Taylor

Strategy

Goals/Objectives

Mission

Vision

Referral/Loyalty

CallerPerceived Value

OverallQuality of

Products &Services

Cost of Acquisition

& Usage


The Contact Center

The Contact Center

14

Frederick Taylor

Strategy

Goals/Objectives

Mission

Vision

Referral/Loyalty


OverallQuality of

Products &Services

Cost of Acquisition

& Usage


Calls handled per hour

The Contact Center

The Contact Center

Talk timeASA

Abandon rateEtc.

Key Performance Indicators – Set 1


Key Performance Indicators – Set 2

What percent of calls are placed on hold properly by the SPECIALIST?What percent of calls contained questions that were never answered by the specialists, causing call backs?What percent of calls contain hold patterns that cause abandons and call backs?What percent of calls would cause the caller to call back hoping for a different specialist? What percent of calls contain irrelevant conversation thereby increasing talk time?What percent of calls contain tragic phrases?

15


Difference between #1 and #2 – CONSENSUS REALITY

# 1 From the inside out, drives consensus reality

# 2 From the outside in – the callers perception! (the specifics of what the caller encounters and the criteria the caller uses to evaluate the center's performance)

16

17The 211 Specialist Monitoring

Program Objectives

Performance Improvement

Outcomes

Patent Pending, 2007

Improved Caller

Satisfaction

Reduced Cost of Center Operation

Improved Allocation

of Technical

Resources


18

The 211 “Mystery Caller” Benchmark Program

* Built on Fraser’s SETS Program (Service Excellence Tracking )

* Fraser’s model based on multicollinearity and regression modeling of over 10,000 call center observations

* Refinement of the model, iterative between Fraser and 211


19

The 211 “Mystery Caller” Benchmark Program

Fraser’s SETS Research Results: 5 major categories

The Greeting (The Initial Contact)Listening, Confirmation, Questioning (Assessment and Clarification)Appropriate Solution (Information and Referral Giving)Professionalism (Communication Techniques)Closing (Closure)

Total of 50 specific criteria (results in a total of 2,500 observations for the Benchmark)

36 specific attributes of measurement Augmented with 211 specific requirements developed qualitatively with 211

20Specialist Monitoring Using the Voice of the Customer Through Mystery Calling

Overall Quality of Specialist Performance


Contact The Initial

Greeting and Building Rapport

Assessment and

Clarification

Information and Referral Giving

Communication Techniques

Build the Relationship with Professionalism

Closure


How was the pilot executed


Participating 2-1-1 Centers

1 - Broward County 2-1-1 2 - NJ Partnership 2-1-1 3 – United Way of Escambia County 2-1-1 4 – United Way of Northeast Louisiana 2-1-1 5 – WIN 2-1-1 6 – United Way of Greater Atlanta 2-1-1 7 – United Way of Greater Cleveland 2-1-1 8 - 2-1-1 LA County 9 – United Way Association of South Carolina 10 - NJ Partnership 2-1-1 (second location) 11 – 2-1-1 Palm Beach 12 – Michigan 2-1-1 (Gryphon Place)


23The Fraser Assessment Scoring Methodology

Each Attribute scored 0 or 1 Most attributes of call scored are objective, i.e.

Thank the caller for calling 211 of ________ Did the Specialist really listen to the caller’s

statements/questions? Did the Specialist paraphrase the purpose of the

call? Did the Specialist handle the proper number of

needs as noted on the scenario used? Did the Specialist avoid the use of jargon and tragic

phrases, unless explanation of the jargon was given?

24

Magic and Tragic PhrasesMAGIC Phrases (a partial list)

•I can help you with that right now•I can look into that for you•I can take care of that for you •I’d be glad to help you•I’m sure I can do that•That would upset me too if that happened to me•Mr. XXX. It would help me get your answer quicker if I knew your order #, do you have that available?•I’m glad I was able to help

25

Magic and Tragic PhrasesTRAGIC Phrases (a partial list)

•Please hold•Give me your ZIP•Hang on a minute•What’s your problem? •If you had read your confirmation more closely•That would be too hard for us to administer•Let me tell you what you haven’t considered•You don’t understand our problem•We have too many calls holding right now for that•You have to•Let me be honest with you

Outbound calls based on 23 scenarios covering• Housing and Utilities-34%• Food and Meals – 26%• Information Services• Income Support• Individual& Family &Community support• Legal, Consumer &Public Safety• Health Care• Mental Health & Addictions• Clothing • Transportation – 2% • Other Govt./ Economic Services – 10%• Employment• Education• Disaster Services• Volunteers and Donations • Arts, Culture

Outbound calls based on singular needs and unspoken needs

SCENARIO Caller:43 YOMale (or Female)No childrenZip Code - Needs – 2Need type - 3 “Hi. I live in (CITY FROM 211 AGENCY BEING CALLED) and my electric is turned off. The Electric Company isn’t being very nice because I can’t pay a bunch of money to get hooked back up. They want $250 to reconnect and I can only pay about $100. I just lost my job and I just paid them $150 last month and they still disconnected me. I think I owe $400 total, can you guys help me get my electricity back on?


SCENARIO CallerMale or Female26 YOLives in city that agency being called is inHas had one leg amputated due to an auto accidentZip Code - Needs – 1Need type - 12 I need help with moving. I have a lot of stuff and don’t have a truck and can’t afford to rent one. Is there anything you can do for me?


SCENARIO Caller23 YO FemaleNot married2 children, 4 and 7Lives in city that agency being called is inZip Code - Needs – 2Need type - 4 I need to find a food pantry to get some food to feed my kids but I can’t leave my job during the day. Are there any that are open at night?


The Agenda

The 211 “mystery caller” benchmark program Consensus reality, aka “Why does quality assurance

matter” Lessons learned from the 211

“mystery caller” benchmark program The path forward


Red Calls• Specialist places caller on hold immediately upon connection

• Specialist uses profanity

• Specialist sets headset aside

• Specialist makes any disrespectful remark toward or laughs at the caller

• Specialist makes any disrespectful remark toward 211

• Specialist makes any disrespectful remark toward another agency

• Specialist talks to neighbor during the call

• Specialist asks the caller to call back (due to specialists or 211). If caller not prepared with information, do not make a red call.

• Specialist is eating, smacking gum or has hard candy




Contact The Initial Greeting and

Building Rapport

Assessment and Clarification




Closure

65%

34%

50%

76%

0%

36%

76%

46%

63%76%

36%

52%66%

37%

48%67%

35%

54%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

33

Overall

Quality

Contact

Assessme

nt and

Clarificati

on

Information

and Refer

ral Givin

g

Communicatio

n Techniques

Closing

Standard Minimum 80%

Overall Calls Score

High Low Benchmark Total

Specialist Monitoring Using the Voice of the Customer Through Mystery Calling

34



Contact The Initial

Greeting and Building Rapport Assessment and

Clarification




Closure

100%

0%

27%

100%

0%

70%100%

0%

52%

80%

0%

18%

80%

0%

15%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

35


Contact

Thank caller for

calling Agency

Identify him/

herself by name

Portray a warm,

welcoming tone

Personalize call

Validate the ZIP Code


36



Contact The Initial


Assessment and

Clarification




Closure

Service Excellence Tracking“Going Beyond Standard”

100%

20%

87%100%

80%

95%

0%0%

0%

80%

0%

35%

100%

0%

56%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

37

Really listen

No inappropriate interruptions

“I will help” or

some variation

Para-phrase caller’s

question/ concern

Used open and closed

end questions

Assessment & Clarification



38



Contact The Initial


Assessment and

Clarification




Closure


100%

60%

83%100%

40%

70%

100%

40%

75%60%

20%

25% 20%

0%

5%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

39

Response clear and

understandable

No further

prompts

Handled proper

number of needs

Appear knowl.

and accurate

Did the AC repeat

spelling of addresses / vital info

Information Giving





Contact The Initial


Assessment and

Clarification




Closure


100%

20%

58%

100%

0%

35%

100%

0%

15%

100%

80%

86% 80%

0%

23%

00

80%

0%

18%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

41

Avoid the inappropriate use of “we”

Ask permission

to gain more

information

Apologize for repeats

Avoid Jargon

Controlled Silence Gaps



Diffused Anger

Expressed Empathy

0 indicates criteria was NA in all calls scored


100%

40%

62%

100%

20%

68%80%

20%

50%

100%

60%

78%100%

0%

27%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

42

Correct pronunciation,

enunciation and volume

Speak clearly

Positive, upbeat

tone




Speak courteously

Use the caller’s

name twice during the

call




Contact The Initial


Assessment and

Clarification




Closure

100%

20%

68%

100%

0%

7%

100%

0%

42%

100%

80%

95%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

44

Attempt to collect demos.

Demos at correct time Other

Issues?

Closure


Did the agent avoid

Irrelevant conversation



100%

60%

77% 80%

0%

18%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

45

Closer to a resolution at the end of the call

Thank the caller for calling

agency or 211

Closing – Overall Centers Report



46

The Filter

Strategy

Goals/Objectives

Mission

Vision

Referral/Loyalty


OverallQuality of

Products &Services

Cost of Acquisition

& Usage


The Caller

The Organization

The Contact Center

The Contact Center


47The 211 Specialist Monitoring

Program Objectives

Performance Improvement

Outcomes


Improved Caller

Satisfaction

Reduced Cost of Center Operation

Improved Allocation

of Technical

Resources




Contact The Initial


Assessment and

Clarification




Closure

65%

34%

50%

76%

0%

36%

76%

46%

63%76%

36%

52%66%

37%

48%67%

35%

54%

-100%

-80%

-60%

-40%

-20%

0%

20%

40%

60%

80%

100%

HighLowTotal

49

Overall

Quality

Contact

Assessme

nt and

Clarificati

on

Information

and Refer

ral Givin

g

Communicatio

n Techniques

Closing


Overall Calls Score

High Low Benchmark Total


The Agenda

The 211 “mystery caller” benchmark program Consensus reality, aka “Why does quality

assurance matter” Lessons learned from the 211 “mystery caller”

benchmark programThe path forward


BUILDING A QUALITY ASSURANCE

PROGRAM FOR 2-1-1 CENTERS USING ‘VOICE OF THE CUSTOMER’

RESEARCH BARRY MANERS, MANAGING DIRECTOR, THE FRASER GROUP

LISA AUSTIN, DIRECTOR OF 2-1-1 STRATEGIC ENHANCEMENTS AND DISASTER RECOVERY UNITED WAY WORLDWIDE

building a quality assurance program for 2-1-1 centers using ‘voice of the customer’ research

Documents

fraser groupvoice

fraser group lisa austin

performance benchmark

united way of greater

united way of escambia

disaster recovery united

quality assurance program

customer behavior