not all customer experience variation is equal - use control charts to identify actual changes in...

Kinesis CEM, LLC

Not All Customer Experience Variation is Equal: Use Control Charts to Identify Actual Changes in the Customer Experience 

https://blog.kinesis-cem.com/2015/04/09/not-all-customer-experience-variation-is-equal-use-control-charts-to-identify-actual-changes-in-the-customer-experience/

Eric Larse is co-founder of Seattle-based Kinesis, which helps companies plan and execute their customer experience strategies. Mr. Larse can be reached at elarse@kinesis-cem.com.

http://www.kinesis-cem.com

kinesis-cem.com 206.285.2900 info@kinesis-cem.com

Not All Customer Experience Variation is Equal:

Use Control Charts to Identify ActualChanges in the Customer Experience

Be it….

a survey of customers…

Variability is Common

mystery shops…

Variability is Common

social listening…

Variability is Common

or…

other customer experience

measurement…

Variability is Common

a certain amount of random variation in the data is normal.

Variability is Common

Managers need a means of

interpreting variation to evaluate if the

customer experience is truly changing, or

if the variation is simply random.

The Problem

Control Charts

One Solution

Statistical Tool to Measure

Variation Control Charts

Consider this example of mystery shop results:

For Example

May June July Aug Sept Oct Nov75%

77%

79%

81%

83%

85%

87%

89% Average Mystery Shop Score

Was July a bad month, or a random variation?

Does the improved performance in October and November represent a true change in the customer experience?

QuestionsMay June July Aug Sept Oct Nov75%

77%

79%

81%

83%

85%

87%

89% Average Mystery Shop Score

Control Limits

To answer these questions we need to set upper control

limits (UCL) and lower control (LCL)

limits.

Control Limits

Where:x = Grand Mean of the scoren = Mean sample sizeSD = Mean standard deviation

The following equations set control limits with a 95% confidence:

Control Limits

Applying these limits to our hypothetical mystery shop scores,

produces the following UCL & LCLs:

May June July Aug Sept Oct Nov75%

77%

79%

81%

83%

85%

87%

89% Average Mystery Shop Score

Questions

Now, we can answer our questions:

May June July Aug Sept Oct Nov75%

77%

79%

81%

83%

85%

87%

89% Average Mystery Shop Score

July was, in fact, a bad month.

While October is still with the threshold of random variation, November showed significant improvement.

Control charts give managers a meaningful way to determine if any

variation in their customer experience measurement

reflects an actual change in the experience as opposed

to random variation or chance.

Control Charts