impact analytics in practice - experience ’20 › ... › macys_exp17.pdf · impact analytics in...

Impact Analytics in PracticeCheryl Berinato, VP Market Research, Macy’s

Karen Hudzinski, Manager Market Research, Macy’sPeter Kriss, Lead Research Scientist, Medallia

2

Customer Experience at Macy’s

Macy’s My Customer Engagement Program measures customer experience across main touchpoints…

‘Test and Learn’ is not new to Macy’s, but the magnitude and speed of this process have increased.

3

How We Analyze Impact TodayWhat?

Standard monthly analysesAd hoc projects

When?Monthly reporting cadence

YoY primary benchmark

Who?Key CX stakeholders:

Macy’s Stores & macys.com, Furniture & Mattress, Wedding & Gift Registry,

Macy’s Credit Customer Service, Macy’s Backstage, Bloomingdale’s

Omnichannel & Outlet

What works well?Collaboration from stakeholdersData as yardstick of initiative successEngagement from all areas of the organization

What are the challenges?Limited sample size for smaller programsMultiple data sources within MCY & BLM

4

What About Your Organization?

Who are the primary owners of improvement initiatives?

a) Frontline leadersb) Mid-level operational leadersc) Senior operational leaders of each functiond) A centralized teame) Other

What level of knowledge does the CX team usually have about specific initiatives?

a) No insight into what they areb) Aware but only at high levelc) Knows the details well but not

involved in executiond) Directly involved in execution

5

What About Your Organization?

Compare and contrast your organization with others at your table:

Who evaluates whether an initiative is successful?

What types of success metrics are most critical?

What level of analytical rigor is expected?

Choose a notable similarity or difference to share with the room

6

Our Impact Analytics Journey

2015 2016 Today

“Innovation at Scale” Research SciencePartner Program

Results of Prototype

7

A Collaborative Prototype ProcessWeekly conversations with Peter to discuss:

Results and implications of tests run throughout the weekApproaches to dig deeper into noteworthy results

Different or new initiatives to testBarriers to tests we wanted to run but could not

8

Fine Jewelry & Watch Pilot – Impact on NPSPurposeto compare group impacted vs. group not impacted over time to measure what that impact is

Impact scoredifference between the impact and comparison groups, taking into account how much they differed before the pilot

NPS improvementof 24% among jewelry/watch customers attributable to the pilot

+ ##.##

+##.####.##

9

Fine Jewelry & Watch Pilot – Impact on Spend

+ ##.##

+##.####.##Average jewelry/watch

purchase size saw a double-digit percentage increase attributable to the pilot

10

Top Doors Initiative – Impact on NPS

NPS improvement of 1.8%among all customers who purchased in Top Doors pilot stores attributed to the changes

+ ##.##

+#.##

#.##

11

How Action Impact is Valuable

Conceptually valuable

Different lens, fresh perspective

Convenient platform for advanced analyses

Practically valuable

Embedded into Medallia portal

Visual representation

Level of protection

12

Implications and Learnings

Implications for the organization

“Test and learn” phase is not just a phase

Innovation is a necessity

Implications for Customer Engagement team

Addition to quarterly, seasonal, or annual recaps

Quick pulse on pre/post analyses

Indicate when customers begin to give credit for improvements

13

What We Learned About theImpact Analysis Process

Iterative and exploratory Opportunity to go deeper

14

What We Learned About the Potential Value

Prototype Client X

It quickly and easily answers a lot of the questions that we probably spend way too much time trying to figure out the answers to.

“”

Prototype Client Z

It helps us on our own, without assistance from our global analytics team, get a quick and accurate read on the impact … to help us make better decisions about what to do next.

“”

Making analysis easier is good…

…but the key value is supporting decisions

15

Quantifying the impact of initiatives is a strategic imperative

(and it’s never been easier)

16

Q&A