remote customer feedback

presents

@UXdna

without getting out of the building

How to get out of the building

@UXdna

What I won’t talk aboutHow to use different remote testing tools

Advantages and pitfalls of each type (moderated, unmoderated, automatic, live, etc.) For this you can read Nate Bolt’s amazing book ‘Remote Research’ or watch his presentation here: http://www.slideshare.net/boltpeters/remote-research-the-book-the-talk

http://www.slideshare.net/boltpeters/remote-research-the-book-the-talk




@UXdna

This is about

A practical guide on how to interview and observe customers remotely

@UXdna

PREPARE & ANTICIPATE

@UXdna

A Spanish company wants to launch an online restaurant listing and booking service in Brazil.

Example

@UXdna

The process

3. How?

How can you get there, step by step

2. Who?

Who you need to talk to/test with?1. What?

What do you need to learn from it?

@UXdna


@UXdna

1. What?

Interview Show + Interview Beta/Online product Prototype


@UXdna

1. What?


SCRIPT OR TOPIC MAP

SCRIPT OR TOPIC MAP

JOURNEY +

SCRIPT OR TOPIC MAP

JOURNEY +

A PROTOTYPE+

SCRIPT OR TOPIC MAP


@UXdna

1. What?


SCRIPT OR TOPIC MAP

SCRIPT OR TOPIC MAP

JOURNEY +

SCRIPT OR TOPIC MAP

JOURNEY +

A PROTOTYPE+

SCRIPT OR TOPIC MAP

VOICEWEBCAM

VOICEWEBCAM

SCREEN SHARING

VOICEWEBCAM

SCREEN SHARING

VOICEWEBCAM

SCREEN SHARING


@UXdna

An Axure prototype of a 3 page journey

Example

Will need to build and test prototype

Will need screen sharing

Will need voice

Will need to define a scenario to users

Will need to explain to users that this is a prototype, not a real site

@UXdna

A. Location

2. Who?RECRUITMENT

What do you need to know about them to make them good

candidates for a test/interview?

Where are the people you will

need to interview/test with?

What will you need to say to get their attention and how

to do it

Where can you find them?

B. Behaviours C. Channels D. What to say?

@UXdna

A. LOCATION

2. Who? Recruitment

LANGUAGESTIME ZONEACCESS

TO TECHNOLOGYVIEWERS

Are the potential participants on my time zone?

What are the times of the day I will be able to talk to them/test?

Do I speak the local language?

Do I need to hire translation?

Do I need to recruit participants who speak English?

What is the country’s internet connection speed like?

Skype says 300 kbps but you need at least 512 kbps

Speedtest.net

Will I be able to stream to viewers as well?

Where are the people you will need to interview/test with?

@UXdna

Example

@UXdna

Example

PORTUGUESE5 HOURS BEHINDOK SPEED

@UXdna

2. Who? Recruitment

TIGHTLY DEFINED SET OF BEHAVIOURS

LOOSE SET OF BEHAVIOURS

I need to talk to teenagers who

have an Android phone and

download at least 3 new games a

week, every week

I need to talk to people who go out dining and check for good

restaurants online

B. BEHAVIOURS What do I need to know about them?

@UXdna

We needed to know they were people who are likely to be our current hypothesis customers, i.e. anyone with a certain level of income (+ 5 minimum wages), who

habitually looked for advice about places to eat online

Example

TIGHTLY DEFINED SET OF BEHAVIOURS

LOOSE SET OF BEHAVIOURS

@UXdna

2. Who? Recruitment

Call centre contactsList of emails from current customers

Intercept as they use website

Publish an ad or post in a forum

Easier Difficult

LIST OF EMAIL ADDRESSESor PHONE NUMBERS

C. CHANNEL Where can I find them?

@UXdna

We published a job position for ‘site tester’ in free job posting websites and ‘Craigslist’ style

lists

Example

Call centre contactsList of emails from current customers

Intercept as they use website

Publish an ad or post in a forum

Easier Difficult

NOTE: It is important to get an idea of what sites people use in the country you are testing

@UXdna

2. Who? Recruitment

Create a screener or list of questions with which you can filter out unsuitable participants

D1Write an email through which you will send the survey link

D2Collect survey results, filter and select

D3

SURVEY LINKSURVEY ANSWERS

D. WHAT TO SAY?

@UXdna

2. Who? Screener questionnaire

List 3 characteristics someone should display to make them likely to be a customer

1List 3 behaviours that make someone unlikely to be your customer

2Based on these factors, write questions that have discrete answers (like ‘yes’ and ‘no’ or multiple choice)

3Don’t ask ‘would you’ questions. Ask ‘what was the last time you...’ questions

4

A GOOD SURVEY MAKES IT EASY TO FILTER

D. WHAT TO SAY? What makes a good screener?

@UXdna

Example

Must live in SP

Must have dined in a restaurant about which they found reviews online in the last 7 days

Must have consulted at least 1 review site

Only seek recommendations from friends

Always go to the same restaurant, don’t like trying new places

Only dine out for very special occasions

What do you usually do...

Would you generally...

Where do you live? (a-d)

When was the last time you visited a new restaurant?

Which, of the sites below, have you visited in the last 7 days?

List 3 characteristics someone should display to make them likely to be a customer

1List 3 behaviours that make someone unlikely to be your customer

2Based on these factors, write questions that have discrete answers (like ‘yes’ and ‘no’ or multiple choice)

3Don’t ask ‘would you’ questions. Ask ‘what was the last time you...’ questions

4

@UXdna

2. Who? Screener questionnaire

MY SURVEYOnly show me answers from people who replied marked the option ‘last 7 days’ to the question

“When was the last time you looked online for restaurant reviews?”

456 respondents = 36 fit our criteria

@UXdna

2. Who? Email contact

Hi,

You are receiving this email because you replied to a job post on [name of website you used here].

My name is [your name here] and I am a researcher/CEO/Founder at [name of company here], ([website link here, if it exists]). We are working on a project about leisure habits in Brazil.

We would like to count on you for a phone interview, but beforehand we would like to know a little more about you. Could you please answer the questions on the following link

(link to survey, very visible)

D. WHAT TO SAY? What makes a good contact email?

@UXdna


Hi,





This is why you are receiving this email from someone you don’t know. You are interested in what I have to say, keep on reading.


@UXdna


Hi,






This is who I am, this is what I am working on and these aremy credentials


@UXdna


Hi,







We are interested in including you in the research, but I need to know if you are the right person first


@UXdna


Hi,







We are interested in including you in the research, but I need to know if you are the right person first

VERY VISIBLE LINK TO SURVEY


@UXdna


We will be in touch if you are selected for the interview. You should be available for a phone call on the [day & date] or [day & date] morning.

We will call you using Skype, and the whole interview shouldn’t last more than 40 minutes. You should provide us with a Skype name or a landline number we can call you on.

Lastly, I would like to let you know that the payment for your time will be made via Paypal, so you must have an active Paypal account.

Thank you already for your interest in participating

Best regards

[your name here]


@UXdna






Best regards

[your name here]

Next steps and confirmation of availability (dates). If you are not available, don’t bother going any further


@UXdna






Best regards

[your name here]


Explanation of what you will need to have: Skype, phone,computer with unrestricted internet connection, etc.


@UXdna






Best regards

[your name here]


Explanation of what you will need to have: Skype, phone,computer with unrestricted internet connection, etc.

How I will pay for your time. If you are not happy, let me know


@UXdna

3. How?

Test your whole setup. What can go

wrong?

What you are testing and how

you are collecting feedback

What can I improve for next time?Show time!

Show time!

A. Set up B. Pilot C. Test D. Post test

@UXdna

3. How?A. SET UP What do I need to prepare for the test?

@UXdna

3. How?


A. SET UP What do I need to prepare for the test?

@UXdna

3. How?


SCRIPT OR TOPIC MAP

SCRIPT OR TOPIC MAP

JOURNEY +

SCRIPT OR TOPIC MAP

JOURNEY +

A PROTOTYPE+

SCRIPT OR TOPIC MAP


@UXdna

3. How?


SCRIPT OR TOPIC MAP

SCRIPT OR TOPIC MAP

JOURNEY +

SCRIPT OR TOPIC MAP

JOURNEY +

A PROTOTYPE+

SCRIPT OR TOPIC MAP

VOICEWEBCAM

VOICEWEBCAM

SCREEN SHARING

VOICEWEBCAM

SCREEN SHARING

VOICEWEBCAM

SCREEN SHARING


@UXdna

Example

Prototype

JOURNEY +

A PROTOTYPE+

SCRIPT OR TOPIC MAP

VOICEWEBCAM

SCREEN SHARINGRECORDING

A scenario so you can situate people:“Imagine it is Sunday morning and you need to

find a place to have lunch with a friend who just arrived in town. You want to try something new”

Prototype was built with wireframes. It was important to let participants know that they

were not going to be looking at a real website, even though buttons were clickable

Themes and questions you would like to discuss with participants.

Do they understand the order of the search results list? Do they see and understand the

ratings ?

Does my set up work?Test all technology works and is compatible

with each other

@UXdna

3. How?

@UXdna

3. How?


What can go wrong, will go wrong

@UXdna

3. How?


What can go wrong, will go wrong

PILOT, PILOT, PILOT!HAVE A FALL BACK PLAN

@UXdna

3. How?

Problem What to check? Fallback during test

B. PILOT What problems should I be trying to mitigate in a pilot?

@UXdna

3. How?

Internet connection:gets really slow or drops completely

•Speedtest.net can give you an idea of the connection speed - or at least let you know if participant has dial speed

•Turn off webcam•Kill other applications •Connections often better at weekends or holidays

Screen sharing:drags a lot or stops completelyor participant can’t start it

•Test out screen sharing•Send ‘how to’ step by step screenshots.

•Check Skype version (4.0 ok, 5.0 better)

•Alternative screen sharing platform (joinme, WebEx, GoToMeeting, Co-pilot)

Can’t hear participantHe can hear you, but you can’t hear him - or the other way around

•Test your mic & speakers•Test their mic & speakers

•Call a landline and still do screen sharing



@UXdna

3. How?



@UXdna

3. How?

No showparticipant doesn’t show up at show time

•Recruit spare participants or ‘floaters’ for each slot allocated for test/interview, who will be waiting in case someone doesn’t show up

•They only get paid if they are needed (you should make this clear when recruiting)

•If both participant and floater fail you, check if other floaters can fill the gap.

Prototype/siteCan participants see the prototype exactly as I planned?Can users access the site?

•Send them a link to the prototype/mock prototype (if you don’t want to show real thing before the test)

•Screen share it! •Send link to a friend first•Ask them to identify something on the screen

•Don’t waste time, get your spare participant in there



@UXdna

3. How?



@UXdna

3. How?

RecordingDoes my recording set up work with the screen sharing and voice?

•Try recording the conversation you have during the pilot session

•WARNING: Silverback is a little odd when webcam + screen sharing!

•Turn off your webcam•Record voice only•Use spare participant•

LanguageI can’t understand what the participant is saying (mine or participant’s language skills aren’t as good as I thought, accent is really difficult to understand)

•Have a conversation with the participant ahead of time and you will be able to gauge how you understand each other

•You shouldn’t get to the actual test with someone who doesn’t fit.

•If you did, and only realise it at the time of test, call in a spare



@UXdna

3. How?



@UXdna

3. How?

Bad recruiting:the participant doesn’t sound like a customer

•WARNING: first impressions can be deceiving, but talking to the participant will give you an idea. You can also ask extra screening questions, or check answers in their questionnaire.

•Spare participant

RapportMain drawback of not being in the room with participant is: How can I make them trust me and tell me stuff?

•It will always be a problem•The pilot helps!•Smile & be nice•Webcam, so they know you are not a machine

•Show your surroundings, if you can (don’t ask them to do the same)

•If at the time of testing participants don’t feel comfortable for some reason, don’t insist, call in your spare!



@UXdna

3. How?

YOU

VIEWER

PARTICIPANT

SKYPE PRO

1. Establish Skype call with viewers (separate username)

2. Turn off their webcam and mic

3. Add participant to the conversation

4. Ask them to turn on web cam

5. Explain who are the other people in the call

6. Begin test!

SKYPE

SKYPE + Screen sharing

C. TEST How to set it up

@UXdna

3. How?

You

Check your own internet connection speed

Check Skype Pro credit ($$)

Check Skype mic

Add participant to Skype contact list, so you know when they are available

Check recording application/device (Highlight, Silverback, notebook) is working

ChecklistC. TEST

Participant

Check participant internet connection speed

Check Skype mic & speakers

Check screen sharing works and they know how to do it

Check they can see your site or prototype well

Check Paypal email address (might be different from the one participant used when replying to ad)

Get their landline & mobile phone number

@UXdna

[email protected]

Thank you!

remote customer feedback

Documents

good need

test prototype

axure prototype

example portuguese

real site

talk abouthow

job position fornote

ad orintercept