![Page 1: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/1.jpg)
Impact Analytics in PracticeCheryl Berinato, VP Market Research, Macy’s
Karen Hudzinski, Manager Market Research, Macy’sPeter Kriss, Lead Research Scientist, Medallia
![Page 2: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/2.jpg)
2
Customer Experience at Macy’s
Macy’s My Customer Engagement Program measures customer experience across main touchpoints…
‘Test and Learn’ is not new to Macy’s, but the magnitude and speed of this process have increased.
![Page 3: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/3.jpg)
3
How We Analyze Impact TodayWhat?
Standard monthly analysesAd hoc projects
When?Monthly reporting cadence
YoY primary benchmark
Who?Key CX stakeholders:
Macy’s Stores & macys.com, Furniture & Mattress, Wedding & Gift Registry,
Macy’s Credit Customer Service, Macy’s Backstage, Bloomingdale’s
Omnichannel & Outlet
What works well?Collaboration from stakeholdersData as yardstick of initiative successEngagement from all areas of the organization
What are the challenges?Limited sample size for smaller programsMultiple data sources within MCY & BLM
![Page 4: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/4.jpg)
4
What About Your Organization?
Who are the primary owners of improvement initiatives?
a) Frontline leadersb) Mid-level operational leadersc) Senior operational leaders of each functiond) A centralized teame) Other
What level of knowledge does the CX team usually have about specific initiatives?
a) No insight into what they areb) Aware but only at high levelc) Knows the details well but not
involved in executiond) Directly involved in execution
![Page 5: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/5.jpg)
5
What About Your Organization?
Compare and contrast your organization with others at your table:
Who evaluates whether an initiative is successful?
What types of success metrics are most critical?
What level of analytical rigor is expected?
Choose a notable similarity or difference to share with the room
![Page 6: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/6.jpg)
6
Our Impact Analytics Journey
2015 2016 Today
“Innovation at Scale” Research SciencePartner Program
Results of Prototype
![Page 7: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/7.jpg)
7
A Collaborative Prototype ProcessWeekly conversations with Peter to discuss:
Results and implications of tests run throughout the weekApproaches to dig deeper into noteworthy results
Different or new initiatives to testBarriers to tests we wanted to run but could not
![Page 8: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/8.jpg)
8
Fine Jewelry & Watch Pilot – Impact on NPSPurposeto compare group impacted vs. group not impacted over time to measure what that impact is
Impact scoredifference between the impact and comparison groups, taking into account how much they differed before the pilot
NPS improvementof 24% among jewelry/watch customers attributable to the pilot
+ ##.##
+##.####.##
![Page 9: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/9.jpg)
9
Fine Jewelry & Watch Pilot – Impact on Spend
+ ##.##
+##.####.##Average jewelry/watch
purchase size saw a double-digit percentage increase attributable to the pilot
![Page 10: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/10.jpg)
10
Top Doors Initiative – Impact on NPS
NPS improvement of 1.8%among all customers who purchased in Top Doors pilot stores attributed to the changes
+ ##.##
+#.##
#.##
![Page 11: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/11.jpg)
11
How Action Impact is Valuable
Conceptually valuable
Different lens, fresh perspective
Convenient platform for advanced analyses
Practically valuable
Embedded into Medallia portal
Visual representation
Level of protection
![Page 12: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/12.jpg)
12
Implications and Learnings
Implications for the organization
“Test and learn” phase is not just a phase
Innovation is a necessity
Implications for Customer Engagement team
Addition to quarterly, seasonal, or annual recaps
Quick pulse on pre/post analyses
Indicate when customers begin to give credit for improvements
![Page 13: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/13.jpg)
13
What We Learned About theImpact Analysis Process
Iterative and exploratory Opportunity to go deeper
![Page 14: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/14.jpg)
14
What We Learned About the Potential Value
Prototype Client X
It quickly and easily answers a lot of the questions that we probably spend way too much time trying to figure out the answers to.
“”
Prototype Client Z
It helps us on our own, without assistance from our global analytics team, get a quick and accurate read on the impact … to help us make better decisions about what to do next.
“”
Making analysis easier is good…
…but the key value is supporting decisions
![Page 15: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/15.jpg)
15
Quantifying the impact of initiatives is a strategic imperative
(and it’s never been easier)
![Page 16: Impact Analytics in Practice - Experience ’20 › ... › Macys_exp17.pdf · Impact Analytics in Practice Cheryl Berinato, VP Market Research, Macy’s Karen Hudzinski, Manager](https://reader034.vdocument.in/reader034/viewer/2022042303/5ece81a6e2314a76e115f8e0/html5/thumbnails/16.jpg)
16
Q&A