Building a User Research Program

I don’t want a UX research program• Not UX, for the sake of UX• Instead, UX for the sake of solving customer

and consumer problems in a way that improves our bottom line

If these are these our goals?• Sustainable growth• Great product reviews• Loyal customers who happily spend more

money with us• Consumers who rave about us

What do you want users to say

Secret answer is D

When the product is amazing…

Why are people promoters?When people think they are amazing, they promote your product.

Why do they tell their friends?People don’t recommend your product because they like your product.

They recommend it because they like their friends.

This is the UX

This is the post-UX UX

Never just design for great UXInstead, design for what happens after the clicking and swiping is done.

Dashboard ux

Selling the American Dream!

Homes Connect is awesome!

I can turn leads into sellers and buyers like

no one’s business.

They want to be excellent at the bigger context

Product design the new way

***

Design for the post-UX UX because…

Source: Bain Capital Research

The numbers are in

• The National Institute of Standard Technology (NIST) in 2002: Cost of fixing one bug at production stage = 15 hrs versus 5 hrs at coding stage.

• The Systems Sciences Institute at IBM: cost to fix an error in production 4-5X cost of discovery in design phase. Up to 100 times more when discovered during maintenance

What are we doing now?

UX happens

Genius design

Principle-based design

• Write design principles• Distill to the entire team• Create pattern libraries• Police and enforce

So, that’s how we do UX now

How do we know it’s good UX?

When it’s (almost) doneThe user doesn’t show up to check out the product until it’s too late:

•QA testing •Beta testing•Customer feedback•A/B testing

How do we fix this?

Destination: Improve bottom line by making users awesome: design for the post-UX UX

UX Strategy

Does this product or feature make customers and consumers awesome at what they do?

Always be asking

Ladder of UX Goodness

In short

Always be asking (the questions users ask)

1. Can I use it?2. Should I use it? 3. Do I want to use it?4. Do I value using it?

People can use the product

User hierarchy of needs

People trust the product is useful to them

People want to use it more than other products

People value using it

The D Word Problem: What does desirable & delightful mean?

1. Tools2. Skill sets 3. Process

In order to ask those questions

Do we know it’s good UX?

• Bring users into process sooner• Users before it even gets to the design

team• When we get really good, we bring them in

even before formal requirements gaterhing

(Definitely bring users in before we get to engineers)

Good UX: research + process

AnalyticsA/B tests

Usability testsPrototyping testsUser testingContextual inquiry (fieldwork)

Design thinkingEmpathyPersona researchJourney Mapping

Competitive usability testsNet Promoter SurveysUser journey mapping

TACTICAL UX

STRATEGIC UX

Good UX: research + process

What and how much?Usability testing costs (in-house)

Remote user testing

For $X/month (+ $1700 fixed costs ) we get:

1. Usable products BEFORE we go live2. User testing cheaper than developer time 3. Method to prioritize fixes and features

Measurable Results:4. Reduce bounce rate5. Increase repeat visitor rate6. Increase engagement metrics

Field Research (all product lines)Design thinking workshops Survey research

What and how much?

For $X/month we get1. Quickly validate ideas at concept phase 2. Validation testing cheaper than developer time3. Resolve design indecision 4. Slash design meeting overhead (e.g., Remax)5. Evidence-based road maps and priority matrices6. Traditional R&D: discover market needs, monetize

Measurable Results:7. Reduce churn8. Increase engagement metrics9. Improve NPS Score10. Improve team throughput

Net Promoter Surveys

What and how much?

For $X/month we get:

1. Reduce customer churn2. Locate pain points that impact bottom line3. Optimize marketing spend: turn neutrals into

promoters for free marketing4. Prioritize $ customers/product roadmap for best ROI5. Valuable consumer data on the consumer journey –

worth money to advertisers/third parties/partners

For toal of $X/month ($X/yr)

1. Usability Testing Program2. Validation Testing (entire product and SEO team)3. Design thinking training, process improvements4. Field research training, process improvements5. Net Promoter: 2x/year relationship survey 6. Net Promoter: Ongoing transaction surveys around lead conversion, MyHomes signup, Connect Onboarding6. Quantitative surveys of customer and consumer behavior

Let’s talk

What other measurable results?

• Aside from the specific tracking metrics listed, what other benefits are there?

1. Decreased development costs: we build the right thing

2. Increased leads3. Improved lead quality4. Increased repeat visitor rate5. Improved number of page views6. Improve NPS = build our fan base7. Higher loyalty scores = lower marketing costs

Measurable results - consumers

1. Increases in customer spend2. Higher NPS and Customer EZ Scores3. Deliver more and better quality leads4. Discover market problems = new revenue

opportunities5. Reduced IT development costs

Measurable results - customers

1. Focused: Customer-centered, small business ethos, artisanal

2. Urgent: validate ideas early, pivot when needed, answer customers needs quickly

3. Collaborative: design-thinking = everyone puts customer satisfaction at center of job

Mature UX fufills our mission

Insights we’ved discovered

• With just a screwdriver and butter knife in our toolbox, we’ve managed to discover and put to use insights useful for the product team

• Better tools will give us actionable insights from marketing, SEO, customer service, sales, design, branding, engineering

Insights – what we’ve discovered so far

1. Map search testing – redacted2. Detail page usability – redacted3. Detail page usability and popups –

redacted4. Feedback form research – redacted5. Intercept research – redacted

More insights in action

6. Mobile testing: users want choice, then smart defaults

7. Mobile testing: search by current location results (redacted)

8. Mobile testing: research on rental and buyer crossover (redacted)

9. Mobile icon research (redacted)

Insights in action

10. Connect CRM: research changed in app messaging

11.Connect: research changed approach for the new agent persona

12. Survey results: insights into important of imagery (redacted)

13.Research confirmed existing assumptions about photos

Research findings we still need to act on:

1. Research on agents use of screeners and qualifiers.

2. Research on home buyer and q/a design pattern3. Research on home feature preferences

(clustering and power law curve)4. Research on gender5. Research on five most important features of the

home itself6. Research on consumers and terminology

Surprises (for me)

1. Preponderance of identity issues2. Preponderance of non-traditional singles3. Use of the app during the post-UX UX