your data is great, but does it work for your users
Post on 13-Apr-2017
122 Views
Preview:
TRANSCRIPT
xmlsummerschool.com Slide 1Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 Unported License
summer school
An introduction to user researchVicky BuserInformation Architect
Your data is great, but does it work for your users?16 September 2015
XML Summer School 2015 2
Learning objectives
By the end of this session, you’ll have the answers to the following 3 questions:1. Why should I be interested in user research?2. What tools and techniques are available?3. How can I extract most value from the research?
XML Summer School 2015 3
Part 1: Why does user research matter?
XML Summer School 2015 4
What makes a bad website?
Chances are you’ll say it’s one that makes it hard to find the information you need or difficult to carry out the transaction you want to make.
Sometimes sites hide useful information under improbable menu headings, or none at all.
XML Summer School 2015 5
Imagine a supermarket with no labels
XML Summer School 2015 6
Why does this happen?
• The site doesn’t speak the users’ language• There’s little or no ‘information scent’• The information is organised in ways that don’t
match the users’ mental models of the domain
XML Summer School 2015 7
What are the costs?
• It can prevent someone from doing their job properly
• People waste time and money• They get frustrated• The site’s credibility is damaged• The site doesn’t get repeat traffic• For commercial sites, the company loses money
XML Summer School 2015 8
As Jakob Nielsen says…
“On the web, if a site is difficult to use, most people will leave.”
XML Summer School 2015 9
So how do you prevent this?
• Do some user research!• How many of you have ever been involved in any
user research? What did you do?• The joy of user research is that there are many
tools and techniques you can use to suit the specific constraints of your project
• Doing user research is the thing I’ve always enjoyed most about my job
• Users (people) will always surprise you and act in unpredictable ways
XML Summer School 2015 10
What we’re going to cover
• Why user research is important for everyone• What it is• Tools and techniques• How to avoid some of the pitfalls• How to extract most value from your findings
XML Summer School 2015 11
Why is user research important for you?
XML Summer School 2015 12
http://alistapart.com/blog/post/developing-empathy
As a developer recently blogged…
XML Summer School 2015 13
And it’s important to me because…I’m an Information Architect. It’s my job to organise, label and group content in ways that help people find what they’re looking for.User research is a really important part of getting this right.
XML Summer School 2015 14
https://insidegovuk.blog.gov.uk/2015/07/14/rebuilding-browse-based-on-user-needs/
XML Summer School 2015 15
Why user research is crucial in IA
• The way we organise data affects the people who use it
• There’s not much point in organising things in ways that don’t help people find what they’re looking for
• People have different ways of looking for the same information, depending on the context
• Language can be very ambiguous• How you organise things depends on your
audience
XML Summer School 2015 16
XML Summer School 2015 17
How would you classify these?• Sweets?• Confectionery?• Chocolates?• Something else?
XML Summer School 2015 18
Language is tricky
• People use different words to describe the same thing: Courgette/zucchini? Chips/crisps? Aubergine/eggplant? Coriander/cilantro? Sweet/desert/pudding?
• Specialists may use different language to the layperson: Pharmacovigilance/Drug safety? Residency/Living in the UK?
• The same words can be understood in more than one way: mercury, fly, fast, fair
• Language is always changing: Chavs, mentally retarded (examples from BBC Archives)
XML Summer School 2015 19
Context affects how we label things
• IAs try to design labels that speak the same language as the site’s users
• They’re an important part of the information scent - the hints a user gets from words/labels
• Labels must be user centric not organisation centric• Given that there’s always an element of messy
subjectivity with labelling, it’s important to validate them with rigorous user research
XML Summer School 2015 20
http://alistapart.com/blog/post/developing-empathy
Back to the developer…
XML Summer School 2015 21
What do I actually mean by user research?• User research isn’t just usability testing• It’s about learning the whole picture rather than
just what happens on the day • It’s an ongoing activity throughout the product
cycle• It’s investigating and analysing what your users
need to inform your site’s strategy• And then validating or disproving design
assumptions, such as “Practitioners need different browse structures to the general public on GOV.UK”
XML Summer School 2015 22
Yes, you say, but I already know about my users. They’re just like me!• Although it’s very tempting, you can’t assume your
users are like you• You are not your user – you’re far too close to the
data/product/content• It’s all too easy to think that other people think
about things the same way that you do• But you need to test these assumptions and
provide evidence to support or challenge them
XML Summer School 2015 23
Developing empathy
http://alistapart.com/blog/post/developing-empathy
XML Summer School 2015 24
What do we want to learn?
• About users’ mental models: how do people currently understand a topic? What kind of picture have they built for themselves about how a given task is done or organised?
• What do users consider most important?• What are the names and relationships between the
terms people use?• What’s the larger picture of the problem we’re
trying to solve?• What are the users’ needs?
XML Summer School 2015 25
Overall benefits
• People’s emotional needs and motivations are more stable over time than shifting technologies, so having an in-depth understanding of your users will help you adapt more easily to new technology
• User research is good for getting consensus/settling arguments: rather than guessing what your users need, you can find out
• Helps to prioritise features so you don’t waste time developing features users don’t need
XML Summer School 2015 26
Part 2: Tools and techniques
XML Summer School 2015 27
Framework for the overall process
• Define hypotheses/assumptions to test• Explore whether your assumptions are correct and
then modify your tests based on your findings• Act on your findings to improve the site• Continue to experiment and iterate• Ask “How can we always get better?”
XML Summer School 2015 28
Mixing and matching the tool kit
• A wide range of methods are available• Best to mix and match according to need and
combine insights• Here is a selection of some of the most commonly
used techniques…
XML Summer School 2015 29
Some quantitative methods
• Site analytics• Search log analysis• Surveys• A/B testing
XML Summer School 2015 30
Site analyticsThe GOV.UK performance dashboard shows us the most popular content, device usage, seasonal patterns in usage etc.
XML Summer School 2015 31
Site analytics
Great for: • Identifying interesting patterns or behaviours for
more probing follow-up research• Measuring improvements (success metrics)Look for:• Usage patterns, traffic cycles, key user journeys
through the site, drop-off points, device and platform usage
Limitations:• Tells you what users do but not why they do it
XML Summer School 2015 32
Search log analysis
Great for: • Seeing what your users are looking for expressed in
their own language• Determining labels that speak the user’s language• Gathering useful information about what users want
from your site and how they articulate their needsLook for:• Acronyms, jargon, new words, length of search
queries, common misspellings
XML Summer School 2015 33
Search log analysisFor example, looking at terms people are searching for from specific pages (on-page searches in Google Analytics) can highlight problems when users aren’t finding the specific information they’re looking for on that page – e.g. a specific form.
XML Summer School 2015 34
SurveysGreat for: • Reaching a broad audience• Finding out about demographics• Getting lots of data from a large number of usersLimitations:• Surveys need to be well designed to be effective
(survey design and analysis are specialised fields of research)
• Best for eliciting facts rather than a deep understanding of user behaviours
XML Summer School 2015 35
A/B testing
What is it?Also known as “multivariate testing,” “live testing,” or “bucket testing”: a method of scientifically testing different designs on a site by randomly assigning groups of users to interact with each of the different designs and measuring the effect of these assignments on user behavior.
XML Summer School 2015 36
A/B testing
Great for: • Measuring the live impact of design changes on key
business metrics (# orders, registrations etc.)• Researching design patterns for small changes to
transactionsLimitations:• Can be hard to frame as a sensible experiment• Many ways to do it wrong• Creates a focus on short-term improvements, so
shouldn’t be used in isolation
XML Summer School 2015 37
A/B testing exampleGOV.UK compared a new design of the Contact page with the old design.Of 50,000 people who came to the contact page, the new design helped 1000 more of them get contact details.
https://gds.blog.gov.uk/2014/05/09/using-ab-testing-to-make-things-better/
XML Summer School 2015 38
Some qualitative techniques
• Contextual inquiry (discovery phase)• User interviews (discovery phase)• Personas (discovery phase)• Usability testing (design and build)
XML Summer School 2015 39
Contextual inquiry
What happens?Researchers meet with and study participants in their natural environment.
GOV.UK childcare and parenting example: user researchers visited 16 families in their homes and listened to their experiences of becoming a parent.
XML Summer School 2015 40
Contextual inquiry
Great for: • Observing how users interact with information in
their natural settings – better than asking about it• Providing environmental context (useful when
physical materials are involved in the process) Tip:• Observe people doing their daily tasks and ask
them about their pain pointsLimitations:• Can be time consuming to set up and run
XML Summer School 2015 41
User interviews
What happens?Researchers meet with participants one-on-one to discuss in depth what the participant thinks about the topic in question.
XML Summer School 2015 42
User interviews
Great for: • Gathering rich, deep understanding about people’s
information needs• Capturing users’ language and how they refer to contentTip:• Listen to the words people use (jargon, acronyms)Limitations:• Can be time-consuming to set up and run• Sensible to talk to a few people initially and then
consider where to go next
XML Summer School 2015 43
ACTIVITY (20 mins) - interviews
Get into groups of 3 and interview each other about mental models related to cooking and recipes, based on this script I’ve prepared. 1 person ask the questions, 1 person answer the questions and 1 person take notes.• To the interviewer: remember to listen• To the note taker: Try to record as many of the responses as
verbatim as possible, so you can re-use quotes for illustrations
• For the interviewee: Try to be natural and answer the questions honestly. There are no right or wrong answers
Take it in turns so that by the end you have three interview transcripts.
XML Summer School 2015 44
Personas
https://assisteddigital.blog.gov.uk/2014/08/28/assisted-digital-user-personas/
XML Summer School 2015 45
PersonasGreat for: • Simple collaborative exercise to get everyone on the team to
start thinking about users• Bringing the user to life, developing empathy• Focusing on target users• Standing in for real users to guide decisions about design and
functionality Tip:• Base your personas on evidence, the details should be factual –
and get the whole team involvedLimitations:• Not a substitute for talking to real users
XML Summer School 2015 46
Usability testing (not ‘user testing’!)
Great for:• Highlighting usability issues and providing
qualitative insights about the problems• Focus, depth, recordings (if using a lab)Tips:• Ask open, neutral questions, get users to ‘think
aloud’, and use self-directed tasks if possibleLimitations:• It’s an artificial and unnatural environment• Doesn’t tell you how big a problem is
XML Summer School 2015 47
Only 5 participants? Yes, really…
http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/
XML Summer School 2015 48
Pop-up usability testing
XML Summer School 2015 49
GOV.UK childcare example
We did remote un-moderated usability testing with parents to see how well they could find relevant information on GOV.UK in it’s previous structure. We found that parents were visiting 3, 4, sometimes even 5 categories before finding the right one.
XML Summer School 2015 50
Some qual/quant methods
• Card sorting• Tree testing• Diary studies
XML Summer School 2015 51
Card sortingWhat happens?Users are asked to organise items on cards into groups and assign category labels to each group. This helps create or refine a site’s IA by exposing users’ mental models.
https://newsignature.com/articles/card-sorting-techniques-pros-cons
XML Summer School 2015 52
Card sorting
Great for:• Simple, cheap and fun way to learn about how
users group and label information in a specific domain
• Providing insights into users’ mental modelsTips:• Look for consistent groups with similar cards in
them and how users have labelled their groups.
XML Summer School 2015 53
GOV.UK childcare example
We did a remote online card sort of content with 100 parents to see whether the initial groupings we came up with were consistent with the way parents think about them.We looked at how often any two cards were placed together (similarity matrix) and the 3 card sorts that had close similarity to the most sorts. We found that there were no strong groupings around financial help (consistent with the lack of knowledge around financial help that came out in the initial research).
XML Summer School 2015 54
ACTIVITY (15 mins) – card sorting of recipesGet into teams and sort these recipe cards into groups that make sense to you.
Then label the groups using the post-its.
Present your findings to the rest of the group at the end.
XML Summer School 2015 55
Tree testing
What happens?Users are asked to find information in a simplified version of the site’s hierarchy. This tests how well information can be found in the structure.
XML Summer School 2015 56
Tree testing
Great for:• Measuring how well users find items in a defined
hierarchy• Testing labels you’re not sure aboutRisks:• Tests how people look for information, not how
they would classify it (that’s what card sorting does)
XML Summer School 2015 57
GOV.UK childcare example
We tree-tested the categories we came up with after the card sorting to check that they made sense and to work out where content should be duplicated in more than one category.
XML Summer School 2015 58
Diary studies
What happens?Participants are given a mechanism (e.g. diary or camera) to record and describe aspects of their lives that are relevant to a product or service, or simply core to the target audience.Analysing diaries kept by users reveals how mental models can change with time.
XML Summer School 2015 59
Diary studies
Great for:• Tracking ongoing relationships with your users over
timeLimitations:• Diary studies are typically longitudinal and can only
be done for data that is easily recorded by participants
XML Summer School 2015 60
Part 3: Challenges and extracting value
XML Summer School 2015 61
Challenges
• Budget• Planning• Doing • Analysis • Reporting
XML Summer School 2015 62
Budget
“We don’t have any time or money for research!”• It doesn’t have to be slow or expensive• Main costs = people’s time, recruiting and paying
participants, equipment costs• ROI - helps reduce waste on things that don’t work
or aren’t needed • Remember the costs of users not being able to find
what they need…• In reality there’s always some form of user
research you can do even on the smallest budget
XML Summer School 2015 63
Planning
Defining the right problems to solve• Quantitative techniques can help you start • Be clear about the objectives of your researchThink about when you do your user research• As early as possible once you have clear goals• Match the right technique to the right project
stage• Write a brief for each activityThink about how the data will be analysed (metrics) and allow plenty of time for the analysis phase
XML Summer School 2015 64
Doing: recruiting your participants
• You need a representative sample• Do the recruitment yourself or outsource it• Recruit as far in advance as possible• Users need incentives (money, vouchers, cake!)• Allow for ‘no shows’How many moderators?• Good to have 2 people: 1 for note-taking• If you can take your team to observe the sessions
that’s great
XML Summer School 2015 65
Doing: on the day
Consent and ethics• You’ll need to get participants’ permission to
record and their consent to use the data you collect (best to only share this internally)
• Allow the participant to opt-out at any time• Respect and listen to your participantsAsking the right questions• Ask open ended questions that encourage
participants to ‘think out loud’• Instead of asking for preferences or opinions, try
to observe what people do
XML Summer School 2015 66
Analysis
• The process than transforms your research data into deliverables that can make an impact
• Don’t skimp on this stage• Go back to the goals you defined in your planning stage
and how you were going to measure success • Don’t worry too much about statistical significance• Look for patterns and insights with post-its and affinity
sorting – ideally as a team exercise• Be realistic about the confidence you can assign to your
results – you can reduce risks but not remove them completely
XML Summer School 2015 67
Reporting
• User research is only as good as the impact it has, so it’s important to communicate your findings
• Use quotations and videos as much as possible in your presentations (remembering to respect users’ anonymity)
• Include subtitles in your video clips if you can• Include stories about what people actually said• Show photos if you have any• You could even use comic strips• Use wall space for your artifacts (GDS style)
XML Summer School 2015 68
Wrap-up
XML Summer School 2015 69
Summary
• Talk to your users regularly about their needs• User research doesn’t always happen in a lab• Identify your target audience• Define the problem before trying to solve it• Don’t assume you are the same as your users • Use research to validate hypotheses• Design decisions should be based on evidence • Get all the team involved if you can• User research is ongoing • Best to test ideas early and often
XML Summer School 2015 70
Resources
Steve Krug’s books:-Don’t Make Me Think-Rocket Surgery Made EasyOnline resources:https://userresearch.blog.gov.uk/http://www.slideshare.net/GDSUserResearchhttp://www.gv.com/library/user-research
XML Summer School 2015 71
Final thoughts
What difference would user research make to the products you’re working on?
Remember…• The costs of users not finding what they need or
struggling to use a site can be high and…• Talking to users is usually fun and always
interesting!
XML Summer School 2015 72
Here’s something for you to try
Friends and family ‘guerilla’ usability testing• A fast, easy, convenient technique• Show a site, a prototype, a sketch…• To: friends and family, colleagues (who don’t work on
the project), people in cafes etc• Ask them about it and observe what they do• Get immediate feedback with minimal overhead• Develop a better understanding of which parts of your
site need improving **CAVEAT: However, where possible you’d want to use actual representative of your audience (see http://www.gv.com/lib/is-it-a-good-idea-to-test-my-product-with-friends-and-family)**
summer school
xmlsummerschool.com Slide 73Licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International License
If you want to contact me here’s my email: vicky.buser@gmail.com
@vickybuser
Thank you!
top related