roundtable chris ezekiel, founder rick britt, vp of ai ... · product portfolio. proprietary &...

ROUNDTABLE

June, 2018

Chris Ezekiel, Founder

and CEO, Creative

Virtual

Rick Britt, VP of AI,

CallMiner

Troy Surdick, Senior

Product Manager,

NICE Nexidia

Artificial

Intelligence in

the Contact

Center

www.creativevirtual.com

CRMXchange Tech Tank

Artificial Intelligence in the Contact Center

Chris Ezekiel, Founder & CEO

@chrisezekiel @creativevirtual

How important is customer experience in making purchase decisions?

Source: PwC Future of Customer Experience Survey 2017/18

Gartner predicts by 2020:

25% of customer service and support operations will integrate virtual customer assistants technology across engagement channels, up from less than 2% in 2017.

Source: Market Guide for Virtual Customer Assistants, Gartner, 2017

Where should my contact center focus?

Centralizing knowledge management

Integrating chatbots & live agents

Combining AI & human input

There are huge benefits to using AI within the CX space, but it requires a solid foundation in knowledge management.

The future of customer engagement lies in humans and machines working in harmony.

Pure AI is not the right solution on its own for providing customer service and support.

Live Demos

▪ Transport for NSW (Web, Facebook, Facebook Messenger, Amazon Alexa)

▪ HSBC Commercial Banking

▪ V-Person Live Chat™ & V-Portal™

RSPCA: consistent

information across web, contact center, mobile

Update Cycle: Combining AI & human input

Learn: System ‘learns’ potential new customer behavior.

Approve: Human editor approves AI suggestions.

Deploy: Updated knowledgebase is

published with improved understanding.

The focus moves from trying to retain knowledge to building better relationships with customers.

The contact center benefits from:

▪ Lower costs

▪ Reduced staff turnover

▪ More engaged, skilled and happier agents

Get in Touch with Me

By email:

[email protected]

On Twitter:

@chrisezekiel

On the web:


Presenter photo

Presenter photo

Intelligence from Customer Interactions

Artificial Intelligence in the Contact Center -

Tech Tank

Proprietary & Confidential, CallMiner Inc.


‹#›

An interaction analytical engine capable of either real-time (intra-contact) or after-contact processing

For QA, speech analysts & data scientists

For QA, supervisors & agents

For developers, data scientists & partners

For compliance & security officers

For supervisors, agents & executives

post-contact real-time

Analytics app for analyzing trends, discovering root cause, and configuring contact categories & scores.

Performance management portal, providing direct automated feedback to supervisors and agents

Comprehensive APIs for contact/data ingestion, and data extraction for app integration & development.

Remove PCI and sensitive data from call audio and transcripts to help retain compliance.

Real-time monitoring, alerting, agent next-best-action, delivered as an API stream for app integration.

Product Portfolio



‹#›

Starting with Data Prep and Feature Extraction

Interactions are converted to data using the our Eureka Analytics platform based on our Mercury LVCSR and redacted for PCI.

“Thank you for calling ABC Bank.. of your social security number [REDACTED]..”

The file and text are tagged with relevant categorical and semantic data

Thank you for calling ABC Bank. How can I help you?

This is my third time calling! You overcharged me on my last bill. I need to speak with a manager

May I confirm your name, address , and last four digits of your social security number?

I’ve already entered my account information in the IVR! You people are ridiculous!

Proper Greeting

Right Party Contact

EmpathyPaymentLanguage

Dissatisfaction CollectorEffectiveness

PolitenessChurnLanguage

CloseLanguage

Features are extracted as well, metadata, speaker, start time of word, word count, scores, outcome dispositions…etc

https://callminer.com/products/eureka-speech-analytics/

https://callminer.com/blog/phonetics-vs-lvcsr-hood-speech-analytics/



‹#›

Real-time Analytics Platform

❑ Features‒ Real-time audio/metadata ingestion

‒ Real-time transcription

‒ Real-time PCI audio/transcript redaction

‒ Real-time rules processing & alerting API (contextual memory)

❑ API benefits‒ Integration into existing agent desktops

‒ Customized action such as messaging

❑ Integration into Eureka platform‒ Feeds real-time transcripts & alert data

to Eureka platform/analyze/coach

‒ Single admin and content reuse

Event:CallSnippetSimple

{

Start: 243.3,

Snippet: “I’d like to speak with a

supervisor”

}

Event:CallAlert

{

Start: 243.7,

Alert: “Escalation”

} Alert GenerationWeighted rules-based pattern

recognition & alerting



‹#›

www.callminer.com/demo

LISTEN TO YOUR CUSTOMERS. IMPROVE YOUR BUSINESS.

A no cost, “Proof of Concept” AI Audit of your customer interactions- your audio recordings and metadata through our contact analytics platform.

Get a free AI Audit with

http://www.callminer.com/demo?utm_source=crmxchange

http://www.callminer.com/demo?utm_source=crmxchange

http://www.facebook.com/callminerinc

http://www.twitter.com/callminer

http://www.youtube.com/callminer01

http://www.linkedin.com/company/callminer

http://www.callminer.com/blog



‹#›

Elements of an AI prediction

CallMiner Training Data (Generated from Billions of

Category matches)

CallMiner or Partner or Customer created Models

CallMiner Eureka Platform Executes Model for results

In Real Time or Batch



‹#›

First Call Resolution

CustomerSatisfaction

CustomerChurn

Upsell/ Revenue Collection

Practical Prediction and Modeling Applications of AI

Probability that a future interaction will happen because of the

current interaction, and recommend

actions to prevent it

Predict the NPS or CX score intracell based

on sentiment and language

Probability of churn based on in call signals,

sentiment, and semantics

Predicting the ideal environment and timing of a sale,

resolution, or upsell.

Strategic Outcome Models



‹#›

Remaining Data

Hold Out

Training Data

Data Set D

Remaining Data

Hold Out

Training Data

Data Set C

Remaining Data

Hold Out

Training Data

Data Set B

Remaining Data

This becomes our data set to model and the fun begins

Hold Out

Training Data

Data Set A Model Selection

Train & Evaluate the Model

Precision = 𝑇𝑃

𝑇𝑃+𝐹𝑃=

14

14+6=

14

20= .7

Recall = 𝑇𝑃

𝑇𝑃+𝐹𝑁=

14

14+2=

14

16= .875

F1 = 2∗𝑃∗𝑅

𝑃+𝑅=

2∗.7∗.875

.7+.875=

1.225

1.575= . 77

Confusion Matrix A𝑇0 𝑇1

𝑃0 2012 82𝑃1 116 1814

Run it for Live Prediction



‹#›

Enriched Interaction Data through NLP and Eureka

Based on our rich data sets we are finding several powerful use cases in enriched NLP data

Basic Feed Forward NN

Recurrent NN

Convolutional NN

Bag of words & Ngrams

Logistic Regression

Neural Nets have been useful

on our data

• Classification Tasks

• Sequencing data

• Building up data sets

Bag of words and word sequencing

is very useful to map interaction

genomes for cause and effect

LR’s and vector

machines are

powerful tools to

predict, separate, and

enhance features

Interaction prediction



‹#›

As we are evaluating methods strong predictive features have emerged

features NB F1 1 NB F1 0 LR F1 1 LR F1 0 NN F1 1 NN F1 0

count vectorizer 76% 71% 93% 93% 93% 93%

word2vec min 60% 69% 87% 87% 1% 67%

word2vec min + max 82% 81% 87% 86% 80% 74%

word2vec max 71% 64% 85% 86% 70% 70%

word2vec sum 70% 56% 90% 90% 45% 70%

word2vec mean 58% 71% 87% 87% 74% 77%

word2vec clusters meanshift 67% 12% 54% 60% 67% 0%

word2vec clusters gaussian 42% 66% 60% 44% 63% 5%

word2vec clusters birch 69% 31% 69% 31% 69% 31%

word2vec embedded layer 83% 82%

count vectorizer 76% 71% 94% 94% 92% 92%

Due to the size and richness of our interaction datasets we have achieved impressive predictive variables



‹#›

Remaining Data

Hold Out

Training Data

Data Set D

Remaining Data

Hold Out

Training Data

Data Set C

Remaining Data

Hold Out

Training Data

Data Set B

Remaining Data

This becomes our data set to model and the fun begins

Hold Out

Training Data

Data Set A

Model Selection

Train & Evaluate the Model

Precision = 𝑇𝑃

𝑇𝑃+𝐹𝑃=

14

14+6= 14

20= .7

Recall = 𝑇𝑃

𝑇𝑃+𝐹𝑁=

14

14+2=

14

16= .875

F1 = 2∗𝑃∗𝑅

𝑃+𝑅=

2∗.7∗.875

.7+.875= 1.225

1.575= . 77

Confusion Matrix A𝑇0 𝑇1

𝑃0 2012 82𝑃1 116 1814

Run it for Live Prediction



‹#›

Biggest Challenge of Conversational ML is tagged data

Typically, machine learning

models need to be trained on

large amounts of data to ensure

that they are accurate, but for

many problems, that large data

set simply doesn’t exist.

- Google’s Former AI chief

John Giannandrea

“



‹#›

Push button prediction based on wide and deep data is on the near horizon

As an interaction is occurring we soon should be

able to predict the outcome based on the strategic

direction, post interaction provide a series of

predictive scores

Example:

FCR: 87% chance of call back

NPS: 4 unhappy

Churn: 63% chance of attrition

CRMXchangeWebinar

June 2018

AI in Analytics

COMPONENTS OF SENTIMENT DETECTION

• Language models identify positive and negative

words and phrases

• Positive phrases can

off-set negative ones

SPOKEN WORDS

• Indicates positive emotional state

• Pervasive element

• Can prevent false positives

and unnecessary alerts

LAUGHTER DETECTION

• Customer and agent talk at the same time,

interrupting one another

CROSS-TALK

• Intensity

• Pitch

• Jitter

• Shimmer

• Speaking rate

PITCH & TONE

27

awesomethis is ridiculous

No problem

I’m so annoyed

TRAINING THE SENTIMENT MODEL

28

SENTIMENT

PREDICTIVE

MODEL

LARGE AMOUNTS OF

LABELED DATA:

RECORDED CONTACT

CENTER CALLS

+

AFTER-CALL SURVEYS

+

CONTACT CENTER

METADATA

TRAINING DATA SET

VALIDATION DATA SET

80%

20%

NEGATIVE CALLS

POSITIVE CALLS

TRAINING THE MODEL

VALIDATE

ADJUST

SENTIMENT AS PROXY FOR AFTER-CALL SURVEYS

Statistical process to:• Normalize sentiment results to

specifics of your customer

environment

• Verify predictive value (statistical

validation of p-scores, means,

medians, etc.)

• Identify meaningful sentiment

ranges

29

2%2% 28% 40% 23% 7%

-31 -2 0 +1 +30+3

Highly Negative Negative Neutral Positive Highly Positive

30

NEXIDIA NEURAL PHONETIC SPEECH ANALYTICS

YEAR FOM

2005 0.48

2009 0.53

2013 0.54

2014 0.65

2015 0.71

2016 0.73

2018 0.78

Represents 10x

reduction in

False Alarm

Rate due to

Deep Neural

Networks

Embedding AI into Nexidia Analytics

31

• IVR Optimization

• Automatic Categorization

• “Neural Net Queries”

• General Purpose Predictive

Model Creation

• Journey Excellence Score

32

Automatically recommend IVR improvements

IVR JOURNEY OPTIMIZATION

32

33

Machine suggests topics for a set of media based on all available data

AUTOMATIC INTERACTION TOPIC CATEGORIZATION

33

34

“Neural Net Queries”• Categorization based on examples

• Machine learns new category based on examples of that category / not of that category

• Semi-supervised machine learning

TOPIC (QUERY) DEFINITION BY EXAMPLE

Cancel

Premium

Service

NEXIDIA

QUERY

RESULTS

Machine

Learning

INTERACTIONS

Expanding the Use

of AI

36

• Score individual interactions

to identify those that most

meet the criteria of interest

(creating a metric)

• Create many metrics that

can be used elsewhere

• Requires large, well-formed

training data sets

INTERACTION-LEVEL PREDICTIVE MODELING

80 42 11 94 31 73 22 75 81 14 77 64

Machine

LearningData Data

PREDICTIVE SCORES FOR EACH INTERACTION

INTERACTIONS

37

Single metric for cross-journey customer experience

CUSTOMER JOURNEY EXCELLENCE SCORE

Consider all customer

touch-points (not just

in the contact center)

Easily identify

Journeys that have

negative impact to

customer experience

38

Simulate Customer Journey scenarios and predict the impact to JES

PREDICTIVE JOURNEY ANALYTICS

Understand how

changes to journey

touchpoints impact the

customer experience

www.nice.com/analytics

ROUNDTABLE

Q&A

Chris Ezekiel, Founder

and CEO, Creative

Virtual


Rick Britt, VP of AI,

CallMiner

www.callminer.com

Troy Surdick, Senior

Product Manager,

NICE Nexidia

www.nexidia.com

Artificial

Intelligence in

the Contact

Center

roundtable chris ezekiel, founder rick britt, vp of ai ... · product portfolio. proprietary &...

Documents