interaction mining: the new frontier of call center analytics

© 2011 interAnalytics 1

Interaction Mining: the new frontier of Call Center Analytics

Vincenzo Pallotta Rodolfo Delmonte Lammert Vrieling

David Walker

© 2011 interAnalytics 2

Outline

• Call Center Analytics• Automatic Argumentative Analysis for

Interaction Mining• Experiments with Call Center Data• Conclusions

3

CALL CENTER ANALYTICS

© 2011 interAnalytics

© 2011 interAnalytics 4

Call Center Analytics

• Call centers data represent a valuable asset for companies, but it is often underexploited for business purposes because:– it is highly dependent on quality of speech recognition

technology– it is mostly based on text-based content analysis.

• Interaction Mining as a viable alternative:– more robust– tailored for the conversational domain– slanted towards pragmatic and discourse analysis

© 2011 interAnalytics 5

Mainstream Call Center Analytics

Does not unveil real

insights about customer

satisfaction

© 2011 interAnalytics 6

Call Center Analytics: metrics and KPIs

• Agent Performance Statistics: – Average Speed of Answer, Average Hold Time, Call Abandonment Rate,

Attained Service Level, and Average Talk Time. – Quantitative measurements that can be obtained directly through ACD

(Automatic Call Distribution), Switch Output and Network Usage Data.• Peripheral Performance Data:

– Cost Per Call, First-Call Resolution Rate, Customer Satisfaction, Account Retention, Staff Turnover, Actual vs. Budgeted Costs, and Employee Loyalty.

– Quantitative, with the exception of Customer Satisfaction that is usually obtained through Customer Surveys.

• Performance Observation: – Call Quality, Accuracy and Efficiency, Adherence to Script,

Communication Etiquette, and Corporate Image Exemplification. – Qualitative metrics based on analysis of recorded calls and session monitoring

by a supervisor.

© 2011 interAnalytics 7

Four objectives

1. Identify Customer Oriented Behaviors, – which are highly correlated to positive customer ratings

(Rafaeli et al. 2007);

2. Identify Root Cause of Problems – by looking at controversial topics and how agents are able to

deal with them;

3. Identify customers who need particular attention – based on history of problematic interactions;

4. Learn best practices in dealing with customers – by identifying agents able to carry cooperative

conversations.

© 2011 interAnalytics 8

ARGUMENTATIVE ANALYSIS FOR INTERACTION MINING

© 2011 interAnalytics 9

Argumentative Structure of ConversationsDISCUSS(issue) <- PROPOSE(alternative)

1702.95 David: so - so my question is should we go ahead and get na- - nine identical head mounted crown mikes ? {qy} 61a

REJECT(alternative)1708.89 John: not before having one come here and have some people try it out . {s^arp^co} 61b.62a

PROVIDE(justification)1714.09 B: because there's no point in doing that if it's not going to be any better . {s} 61b+

ACCEPT(justification)

1712.69 David: okay . {s^bk} 62b

PROPOSE(alternative)

1716.85 John: so why don't we get one of these with the crown with a different headset ? {qw^cs} 63a

ACCEPT(alternative)1721.56 David: yeah . {s^bk} 63b1726.05 Lucy: yeah . {b} 1727.34 John: yeah . {b}

PROVIDE(justification)

1722.4 John: and - and see if that works . {s^cs} 63a+.64a 1723.53 Mark: and see if it's preferable and if it is then we'll get more . {s^cs^2} 64b1725.47 Mark: comfort . {s}

PROVIDE(justification)1714.09 John: because there's no point in doing that if it's not going to be any better . {s} 61b+

Why was David’s proposal on microphones rejected?

© 2011 interAnalytics 10

Automatic Argumentative Analysis

• Based on the GETARUNS system1.• Clauses in Turns are labelled with Primitive Discourse

Relations: – statement, narration, adverse, result, cause, motivation,

explanation, question, hypothesis, elaboration, permission, inception, circumstance, obligation, evaluation, agreement, contrast, evidence, hypoth, setting, prohibition.

• And then Turns are labelled with Argumentative labels:– ACCEPT, REJECT/DISAGREE, PROPOSE/SUGGEST,

EXPLAIN/JUSTIFY, REQUEST EXPLANATION/JUSTIFICATION.

1 Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogueUnderstanding. Proceedings LREC 2010 (P31 Dialogue Corpora).

© 2011 interAnalytics 11

Evaluation

Correct Incorrect Total Found Precision

Accept 662 16 678 98%

Reject 64 18 82 78%

Propose 321 74 395 81%

Request 180 1 181 99%

Explain 580 312 892 65%

Total 1826 421 2247 81.26%

Precision: 81.26% Recall: 97.53%

ICSI corpus of meetings (Janin et al., 2003)

Delmonte R., Bistrot A., Pallotta V.,Deep Linguistic Processing with GETARUNS for spoken dialogueUnderstanding. Proceedings LREC 2010 (P31 Dialogue Corpora).

© 2011 interAnalytics 12

EXPERIMENTS WITH CALL CENTER DATA

© 2011 interAnalytics 13

Rationale: implement the four objectives

1. Identify Customer Oriented Behaviors, 2. Identify Root Cause of Problems 3. Identify customers who need particular

attention 4. Learn best practices in dealing with

customers

© 2011 interAnalytics 14

The Data

• Corpus of 213 manually transcribed conversations of a help desk call center in the banking domain.

• Average of 66 turns per conversation.• Average of 1.6 calls per agent. • Collected for a study aimed at identifying

customer oriented behaviors that could favor satisfactory interaction with customers (Rafaeli et al. 2007).

© 2011 interAnalytics 15

Identify Customer Oriented Behaviors

• Based on the work of Rafaeli et al. 2006.• Customer Oriented Behaviors– anticipating customers requests 22,45%– educating the customer 16,91%– offering emotional support 21,57%– offering explanations / justifications 28,57%– personalization of information 10,50%

© 2011 interAnalytics 16

Significant correlation with argumentative labels

© 2011 interAnalytics 17

Identify Root Cause of Problems

• Cooperativeness score – a measure obtained by

averaging the score obtained by mapping argumentative labels of each turn in the conversation into a [-5 +5] scale.

• Sentiment Analysis module.

Argumentative Categories Cooperativeness

Accept explanation 5

Suggest 4

Propose 3

Provide opinion 2

Provide explanation/justification 1

Request explanation/justification 0

Question -1

Raise issue -2

Provide negative opinion -3

Disagree -4

Reject explanation or justification -5

© 2011 interAnalytics 18

Top 20 Controversial Topics with average cooperativeness scores and sentiment

© 2011 interAnalytics 19

Cooperativeness of speakers on top discussed topics

© 2011 interAnalytics 20

Identify problematic customers

© 2011 interAnalytics 21

Select a specific customer

© 2011 interAnalytics 22

Visualize a selected call

© 2011 interAnalytics 23

CONCLUSIONS

© 2011 interAnalytics 24

Conclusions

• New Generation Call Center Analytics requires Interaction Mining– Call Center Qualitative metrics and KPIs can be only

implemented with a full understanding of the customer interaction dynamics

• Argumentation is pervasive in conversations.– In order to recognize argumentative acts, advanced Natural

Language Understanding is necessary.• Future work:– Scalability: need to process millions of call per day!– Multi-language: call centers all over the world.

© 2011 interAnalytics 25

The Team

www.interanalytics.ch…find us at