your data is great, but does it work for your users

xmlsummerschool.com Slide 1Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License

summer school

An introduction to user researchVicky BuserInformation Architect

Your data is great, but does it work for your users?16 September 2015

http://www.xmlsummerschool.com/

XML Summer School 2015 2

Learning objectives

By the end of this session, you’ll have the answers to the following 3 questions:1. Why should I be interested in user research?2. What tools and techniques are available?3. How can I extract most value from the research?


Part 1: Why does user research matter?


What makes a bad website?

Chances are you’ll say it’s one that makes it hard to find the information you need or difficult to carry out the transaction you want to make.

Sometimes sites hide useful information under improbable menu headings, or none at all.


Imagine a supermarket with no labels


Why does this happen?

• The site doesn’t speak the users’ language• There’s little or no ‘information scent’• The information is organised in ways that don’t

match the users’ mental models of the domain


What are the costs?

• It can prevent someone from doing their job properly

• People waste time and money• They get frustrated• The site’s credibility is damaged• The site doesn’t get repeat traffic• For commercial sites, the company loses money


As Jakob Nielsen says…

“On the web, if a site is difficult to use, most people will leave.”


So how do you prevent this?

• Do some user research!• How many of you have ever been involved in any

user research? What did you do?• The joy of user research is that there are many

tools and techniques you can use to suit the specific constraints of your project

• Doing user research is the thing I’ve always enjoyed most about my job

• Users (people) will always surprise you and act in unpredictable ways


What we’re going to cover

• Why user research is important for everyone• What it is• Tools and techniques• How to avoid some of the pitfalls• How to extract most value from your findings


Why is user research important for you?


http://alistapart.com/blog/post/developing-empathy

As a developer recently blogged…


And it’s important to me because…I’m an Information Architect. It’s my job to organise, label and group content in ways that help people find what they’re looking for.User research is a really important part of getting this right.


https://insidegovuk.blog.gov.uk/2015/07/14/rebuilding-browse-based-on-user-needs/


Why user research is crucial in IA

• The way we organise data affects the people who use it

• There’s not much point in organising things in ways that don’t help people find what they’re looking for

• People have different ways of looking for the same information, depending on the context

• Language can be very ambiguous• How you organise things depends on your

audience


How would you classify these?• Sweets?• Confectionery?• Chocolates?• Something else?


Language is tricky

• People use different words to describe the same thing: Courgette/zucchini? Chips/crisps? Aubergine/eggplant? Coriander/cilantro? Sweet/desert/pudding?

• Specialists may use different language to the layperson: Pharmacovigilance/Drug safety? Residency/Living in the UK?

• The same words can be understood in more than one way: mercury, fly, fast, fair

• Language is always changing: Chavs, mentally retarded (examples from BBC Archives)


Context affects how we label things

• IAs try to design labels that speak the same language as the site’s users

• They’re an important part of the information scent - the hints a user gets from words/labels

• Labels must be user centric not organisation centric• Given that there’s always an element of messy

subjectivity with labelling, it’s important to validate them with rigorous user research



Back to the developer…


What do I actually mean by user research?• User research isn’t just usability testing• It’s about learning the whole picture rather than

just what happens on the day • It’s an ongoing activity throughout the product

cycle• It’s investigating and analysing what your users

need to inform your site’s strategy• And then validating or disproving design

assumptions, such as “Practitioners need different browse structures to the general public on GOV.UK”


Yes, you say, but I already know about my users. They’re just like me!• Although it’s very tempting, you can’t assume your

users are like you• You are not your user – you’re far too close to the

data/product/content• It’s all too easy to think that other people think

about things the same way that you do• But you need to test these assumptions and

provide evidence to support or challenge them


Developing empathy



What do we want to learn?

• About users’ mental models: how do people currently understand a topic? What kind of picture have they built for themselves about how a given task is done or organised?

• What do users consider most important?• What are the names and relationships between the

terms people use?• What’s the larger picture of the problem we’re

trying to solve?• What are the users’ needs?


Overall benefits

• People’s emotional needs and motivations are more stable over time than shifting technologies, so having an in-depth understanding of your users will help you adapt more easily to new technology

• User research is good for getting consensus/settling arguments: rather than guessing what your users need, you can find out

• Helps to prioritise features so you don’t waste time developing features users don’t need


Part 2: Tools and techniques


Framework for the overall process

• Define hypotheses/assumptions to test• Explore whether your assumptions are correct and

then modify your tests based on your findings• Act on your findings to improve the site• Continue to experiment and iterate• Ask “How can we always get better?”


Mixing and matching the tool kit

• A wide range of methods are available• Best to mix and match according to need and

combine insights• Here is a selection of some of the most commonly

used techniques…


Some quantitative methods

• Site analytics• Search log analysis• Surveys• A/B testing


Site analyticsThe GOV.UK performance dashboard shows us the most popular content, device usage, seasonal patterns in usage etc.


Site analytics

Great for: • Identifying interesting patterns or behaviours for

more probing follow-up research• Measuring improvements (success metrics)Look for:• Usage patterns, traffic cycles, key user journeys

through the site, drop-off points, device and platform usage

Limitations:• Tells you what users do but not why they do it


Search log analysis

Great for: • Seeing what your users are looking for expressed in

their own language• Determining labels that speak the user’s language• Gathering useful information about what users want

from your site and how they articulate their needsLook for:• Acronyms, jargon, new words, length of search

queries, common misspellings


Search log analysisFor example, looking at terms people are searching for from specific pages (on-page searches in Google Analytics) can highlight problems when users aren’t finding the specific information they’re looking for on that page – e.g. a specific form.


SurveysGreat for: • Reaching a broad audience• Finding out about demographics• Getting lots of data from a large number of usersLimitations:• Surveys need to be well designed to be effective

(survey design and analysis are specialised fields of research)

• Best for eliciting facts rather than a deep understanding of user behaviours


A/B testing

What is it?Also known as “multivariate testing,” “live testing,” or “bucket testing”: a method of scientifically testing different designs on a site by randomly assigning groups of users to interact with each of the different designs and measuring the effect of these assignments on user behavior.


A/B testing

Great for: • Measuring the live impact of design changes on key

business metrics (# orders, registrations etc.)• Researching design patterns for small changes to

transactionsLimitations:• Can be hard to frame as a sensible experiment• Many ways to do it wrong• Creates a focus on short-term improvements, so

shouldn’t be used in isolation


A/B testing exampleGOV.UK compared a new design of the Contact page with the old design.Of 50,000 people who came to the contact page, the new design helped 1000 more of them get contact details.

https://gds.blog.gov.uk/2014/05/09/using-ab-testing-to-make-things-better/


Some qualitative techniques

• Contextual inquiry (discovery phase)• User interviews (discovery phase)• Personas (discovery phase)• Usability testing (design and build)


Contextual inquiry

What happens?Researchers meet with and study participants in their natural environment.

GOV.UK childcare and parenting example: user researchers visited 16 families in their homes and listened to their experiences of becoming a parent.


Contextual inquiry

Great for: • Observing how users interact with information in

their natural settings – better than asking about it• Providing environmental context (useful when

physical materials are involved in the process) Tip:• Observe people doing their daily tasks and ask

them about their pain pointsLimitations:• Can be time consuming to set up and run


User interviews

What happens?Researchers meet with participants one-on-one to discuss in depth what the participant thinks about the topic in question.


User interviews

Great for: • Gathering rich, deep understanding about people’s

information needs• Capturing users’ language and how they refer to contentTip:• Listen to the words people use (jargon, acronyms)Limitations:• Can be time-consuming to set up and run• Sensible to talk to a few people initially and then

consider where to go next


ACTIVITY (20 mins) - interviews

Get into groups of 3 and interview each other about mental models related to cooking and recipes, based on this script I’ve prepared. 1 person ask the questions, 1 person answer the questions and 1 person take notes.• To the interviewer: remember to listen• To the note taker: Try to record as many of the responses as

verbatim as possible, so you can re-use quotes for illustrations

• For the interviewee: Try to be natural and answer the questions honestly. There are no right or wrong answers

Take it in turns so that by the end you have three interview transcripts.


Personas

https://assisteddigital.blog.gov.uk/2014/08/28/assisted-digital-user-personas/


PersonasGreat for: • Simple collaborative exercise to get everyone on the team to

start thinking about users• Bringing the user to life, developing empathy• Focusing on target users• Standing in for real users to guide decisions about design and

functionality Tip:• Base your personas on evidence, the details should be factual –

and get the whole team involvedLimitations:• Not a substitute for talking to real users


Usability testing (not ‘user testing’!)

Great for:• Highlighting usability issues and providing

qualitative insights about the problems• Focus, depth, recordings (if using a lab)Tips:• Ask open, neutral questions, get users to ‘think

aloud’, and use self-directed tasks if possibleLimitations:• It’s an artificial and unnatural environment• Doesn’t tell you how big a problem is


Only 5 participants? Yes, really…

http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/


Pop-up usability testing


GOV.UK childcare example

We did remote un-moderated usability testing with parents to see how well they could find relevant information on GOV.UK in it’s previous structure. We found that parents were visiting 3, 4, sometimes even 5 categories before finding the right one.


Some qual/quant methods

• Card sorting• Tree testing• Diary studies


Card sortingWhat happens?Users are asked to organise items on cards into groups and assign category labels to each group. This helps create or refine a site’s IA by exposing users’ mental models.

https://newsignature.com/articles/card-sorting-techniques-pros-cons


Card sorting

Great for:• Simple, cheap and fun way to learn about how

users group and label information in a specific domain

• Providing insights into users’ mental modelsTips:• Look for consistent groups with similar cards in

them and how users have labelled their groups.



We did a remote online card sort of content with 100 parents to see whether the initial groupings we came up with were consistent with the way parents think about them.We looked at how often any two cards were placed together (similarity matrix) and the 3 card sorts that had close similarity to the most sorts. We found that there were no strong groupings around financial help (consistent with the lack of knowledge around financial help that came out in the initial research).


ACTIVITY (15 mins) – card sorting of recipesGet into teams and sort these recipe cards into groups that make sense to you.

Then label the groups using the post-its.

Present your findings to the rest of the group at the end.


Tree testing

What happens?Users are asked to find information in a simplified version of the site’s hierarchy. This tests how well information can be found in the structure.


Tree testing

Great for:• Measuring how well users find items in a defined

hierarchy• Testing labels you’re not sure aboutRisks:• Tests how people look for information, not how

they would classify it (that’s what card sorting does)



We tree-tested the categories we came up with after the card sorting to check that they made sense and to work out where content should be duplicated in more than one category.


Diary studies

What happens?Participants are given a mechanism (e.g. diary or camera) to record and describe aspects of their lives that are relevant to a product or service, or simply core to the target audience.Analysing diaries kept by users reveals how mental models can change with time.


Diary studies

Great for:• Tracking ongoing relationships with your users over

timeLimitations:• Diary studies are typically longitudinal and can only

be done for data that is easily recorded by participants


Part 3: Challenges and extracting value


Challenges

• Budget• Planning• Doing • Analysis • Reporting


Budget

“We don’t have any time or money for research!”• It doesn’t have to be slow or expensive• Main costs = people’s time, recruiting and paying

participants, equipment costs• ROI - helps reduce waste on things that don’t work

or aren’t needed • Remember the costs of users not being able to find

what they need…• In reality there’s always some form of user

research you can do even on the smallest budget


Planning

Defining the right problems to solve• Quantitative techniques can help you start • Be clear about the objectives of your researchThink about when you do your user research• As early as possible once you have clear goals• Match the right technique to the right project

stage• Write a brief for each activityThink about how the data will be analysed (metrics) and allow plenty of time for the analysis phase


Doing: recruiting your participants

• You need a representative sample• Do the recruitment yourself or outsource it• Recruit as far in advance as possible• Users need incentives (money, vouchers, cake!)• Allow for ‘no shows’How many moderators?• Good to have 2 people: 1 for note-taking• If you can take your team to observe the sessions

that’s great


Doing: on the day

Consent and ethics• You’ll need to get participants’ permission to

record and their consent to use the data you collect (best to only share this internally)

• Allow the participant to opt-out at any time• Respect and listen to your participantsAsking the right questions• Ask open ended questions that encourage

participants to ‘think out loud’• Instead of asking for preferences or opinions, try

to observe what people do


Analysis

• The process than transforms your research data into deliverables that can make an impact

• Don’t skimp on this stage• Go back to the goals you defined in your planning stage

and how you were going to measure success • Don’t worry too much about statistical significance• Look for patterns and insights with post-its and affinity

sorting – ideally as a team exercise• Be realistic about the confidence you can assign to your

results – you can reduce risks but not remove them completely


Reporting

• User research is only as good as the impact it has, so it’s important to communicate your findings

• Use quotations and videos as much as possible in your presentations (remembering to respect users’ anonymity)

• Include subtitles in your video clips if you can• Include stories about what people actually said• Show photos if you have any• You could even use comic strips• Use wall space for your artifacts (GDS style)


Wrap-up


Summary

• Talk to your users regularly about their needs• User research doesn’t always happen in a lab• Identify your target audience• Define the problem before trying to solve it• Don’t assume you are the same as your users • Use research to validate hypotheses• Design decisions should be based on evidence • Get all the team involved if you can• User research is ongoing • Best to test ideas early and often


Resources

Steve Krug’s books:-Don’t Make Me Think-Rocket Surgery Made EasyOnline resources:https://userresearch.blog.gov.uk/http://www.slideshare.net/GDSUserResearchhttp://www.gv.com/library/user-research

http://www.sensible.com/dmmt.html

http://www.sensible.com/rsme.html

https://userresearch.blog.gov.uk/



http://www.slideshare.net/GDSUserResearch

http://www.gv.com/library/user-research

http://www.gv.com/library/user-research


Final thoughts

What difference would user research make to the products you’re working on?

Remember…• The costs of users not finding what they need or

struggling to use a site can be high and…• Talking to users is usually fun and always

interesting!


Here’s something for you to try

Friends and family ‘guerilla’ usability testing• A fast, easy, convenient technique• Show a site, a prototype, a sketch…• To: friends and family, colleagues (who don’t work on

the project), people in cafes etc• Ask them about it and observe what they do• Get immediate feedback with minimal overhead• Develop a better understanding of which parts of your

site need improving **CAVEAT: However, where possible you’d want to use actual representative of your audience (see http://www.gv.com/lib/is-it-a-good-idea-to-test-my-product-with-friends-and-family)**

summer school

xmlsummerschool.com Slide 73Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License

If you want to contact me here’s my email: [email protected]

@vickybuser

Thank you!

http://www.xmlsummerschool.com/

mailto:[email protected]

your data is great, but does it work for your users

Internet