Download - Rationalising and Designing Methodologies for New Media Research Gosia Kwiatkowska [email protected]
Designing New Media
Choosing your research method
Quantitative or Qualitative Method or both?
Quantitative Methods
– Designed to ensure objectivity, reliability and ability to generalise
– Test predetermined hypothesis – explanations– Who, how much, how many?– Closed questions– Short answers– Averages, percentages, ranges, means, frequencies– Can generate accurate and precise data – Can test statistical relationship between variable– Can prove whether or not a particular problem exists– Can identify specific characteristics of a population
http://documents.wfp.org/stellent/groups/public/documents/ko/mekb_module_13.pdf
Quantitative Methods
• Strengths– Robust – Objective– Verifiable
• Weaknesses– Out of context – human behaviour, real world
settings are not considered – Any variables left out of data collection are not
used in analysis
Qualitative Method?– User perspectives – Questions: how and why?– Captures perceptions, judgements, meanings,
processes and reasons– Open ended questions, checklist of topics– Hypothesis and follow up questions are generated
during data collection and analysis– Subjective – not easy to objectively verify– Interviews allow to probe – Provides deeper insight – Broader understanding and explanations, views
http://documents.wfp.org/stellent/groups/public/documents/ko/mekb_module_13.pdf
Qualitative Method?• Strengths
– Participatory– Rich, detailed data – Considers users perspectives and the context for their
behaviours • Weaknesses
– Hard work– Time consuming– Smaller sample of users– Not easily verifiable– Not easy to group your responses/categorised
Quantitative or Qualitative or Both?
• Quantitative methods can highlight an issues that could then be studies in depth using qualitative methods.
• Qualitative methods might be used at the beginning of a study to help the researcher to decide what closed questions could be used in the bigger quantitative survey or questionnaire.
Design phases
• Phase 1 - User analysis– Questionnaires/ethnography/interviews
• Phase 2 - Conceptual design– Cards sorts/cognitive walkthrough – flowcharts
• Phase 3 - Testing and implementation– User testing of prototypes– Final design
Phase One- User Analysis
http://www.grameenfoundation.applab.org/challenges/what-innovative-mobile-money-mm-products-can-we-develop-that-are-suitable-for-poor-unbanked-customers-and-viable-for-our-commercial-partners/timeline.html
Tools
• Surveys: type of questions?• Interviews: interview questions? • Focus groups: purpose and objectives,
discussion notes, who, why, where? What did you find out? Which quotes, comments will you use to support your findings?
• Observations: observation checklist• Case studies: use triangulation
Checklist for designing and conducting interviews, focus groups, surveys
• Decide what you need to know.• State why you need to know this information.• What is the best way of obtaining this
information.• Outline your questions.• Check wording of each question. • Decide on the tool for data collection and the
question type.• Refine the questions and order them.• Consider how questions will be analysed.
Checklist for designing and conducting interviews, focus groups, surveys cnt
• Write instruction that might be included in your surveys.• Consider layout and appearance.• Decide on your sample, are they representative.• Prepare a schedule. • Pilot.• Revise, if necessary.• Do your best to avoid bias. • Select who to interview.• Ensure official channels have been cleared. Information
sheets, consents etc• Introduce yourself and explain the purpose of the research
Checklist for designing and conducting interviews, focus groups, surveys cnt
Designing your questionnaires
• Questions types– List– Category– Ranking– Scale– Quantity– Grid– Verbal
List
Summary sheetQuestion 13: What do you use the internet for?
Finding interesting information
Finding pictures
Finding and watching videos
Finding and listening to music
Online learning
Shopping Online gaming
Other
||||| || | ||| || |
|||
17 18 12 21 18 27 26 3
Category questions
• How old are you? ______
What will you do with the responses? Depends on what you want to know?e.g. Average age of your users? If so, what sort
of average or measure of central tendency will suit your purpose: the arithmetic means, the median, or the mode?
The arithmetic mean
• Arithmetic mean = sum or each value/total number of values.
• 26 participants• Ages: 21,22,22,25,25,25,25,25,25,26,26, 26,
27, 28, 29, 30,30,32,32,33,34,35,35,35,35,35• Add ages together = 743• Divide 743 by 26 • The mean = 28.57
Mean
• Group A Ages: 21,22,22,25,25,25,25,25,25,26,26, 26,
27, 28, 29, 30,30,32,32,33,34,35,35,35,35,35• Group B Ages: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21,
21, 22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65,
• The same mean but clear difference between the dispersion of results
The median• Allows us to find the middle valueIf we have an odd number of values If we have an even number of values: take the average of
the middle valuesGroup AAges: 21,22,22,25,25,25,25,25,25,26,26, 26,27,28, 29,
30,30,32,32,33,34,35,35,35,35,35Median=(27+28)/2 = 27.5Group BAges: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21,
22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65Median =22.5You decide which one gives more realistic picture.
The mode
• The most frequently occurring valueGroup AAges: 21,22,22,25,25,25,25,25,25,26,26, 26,27,28,
29, 30,30,32,32,33,34,35,35,35,35,35Modal score = 25Group BAges: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21,
22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65Modal score = 19
Range
Range the difference between the highest and the lowest values measured
• Group A 21-35 range = 35-21 = 14 • Group B 18-65 range = 65-18 = 47
Problem? What about the middle numbers?
Interquartile range• Interquartile range indicates the extent to which data
vary.• Derived from the median, the highest and the lowest
quarter of the measures are omitted and the interquartile range of the middle 50 % of values is quoted
Group A 21,22,22,25,25,25,25,25,25,26,26, 26,27,median point28, 29, 30,30,32,32,33,34,35,35,35,35,35Interquartile range is 25-32 or 7 years
Alternative question
• How old are you?18-29 30-39 40-49 50-59 60+
You can still calculate the arithmetic means by using the mid point of each class interval.
Scales
Verbal (open) question
Recording your data
• Data must be recorded, analysed and interpreted.
• Identify categories.• Prepare summary sheets.• Experiment with different ways of presenting
findings.• Remember all data require interpretation.
Presenting your data and analysis
Phase Two-Conceptual design
Making sense of data Organise data Aggregation (combine/collapse redundant pieces of data) Juxtaposing related pieces of data (putting together) Naming the resulting data clusters Juxtaposing unrelated pieces of data
Exploring alternative solutions Choosing a design
(Safer, 2010, p.95)
Conceptual design-Tools
– Brainstorming– Card sort– Personas– Scenarios– Flowcharts– Use cases
Brainstorming
Card Sort
• Step 1 – separate all information • Step 2 – organise the cards into related areas• Step 3 – label your groups• What if one topic fits into more than one
category?• Test• This will provide basis for your information
architecture, the structure and the design.
Card Sort
Advantages• Can be done before design• Quick and easy• Good way of finding out
underlying structures for your design
• Good way of finding out how the end user organises information
Disadvantages• Difficult to navigate when
you have lots of categories• Limited to the topics that
you put on the cards• Card sort outcomes suggest
structures
Personas
http://geoinformatics.fsv.cvut.cz/gwiki/Developing_web_map_application_based_on_user_centered_design
From Personas to Cognitive Walkthroughs
• Scenarios: typical task, goal, conditions at the beginning of the task, activities in which the persona will engage, the outcomes of these activities.
Scenario
http://csweb.cs.bgsu.edu/maner/domains/Scenario.htm
Flowcharts
Information architecture• Organise information• Show navigational structure• Efficiency• Effectiveness
It should correspond to:• Content• User needs • Our objectives
Garrett, 2010, p94
Use cases
Phase Three-Testing and Implementation
The physical design:– Look – Components– Layout
Tools:– Low-fidelity prototypes– Evaluations– Wireframes– High-fidelity prototypes
Low-fidelity prototyping
• Nielsen’s two types of prototypes – Horizontal– Vertical
http://www.useit.com/papers/guerrilla_hci.html
Low-fidelity prototypes:
• Easy and cheap to make• Flexible-can be changed and re-arranged• Complete enough to give useful feedback about specific
design question/s
http://www.nngroup.com/reports/prototyping/video_stills.html
Nielsen Norman Group
Things to consider in order to define the scope of the prototype and the focus:
• What feedback do you need at this point in the design process?• How much of the design should you prototype?• Should you cover all of the areas but without great detail (breadth vs.
depth)?• Should you cover one area in great detail?
Nielsen Norman Group
Evaluation
Begin evaluations early in the design process.
• Evaluation is an integral part of the development process and can take the form of an informal walkthrough or a more structured heuristic evaluation.
• Formal usability testing can begin once a prototype has been developed.
Wireframe-blueprint of a product
Wireframes cnt
• Wireframes help to create template layouts that can be used to impose a consistent structure throughout the interface
http://www.gliffy.com/examples/wireframes/
Service blueprint
• Similar like wireframe for digital product, service blueprint presents service moments and service string. E.g. car wash
• Brainstorm possible service moments• Sketch/photograph/single storyboard frame
each moment• How each service moment affects service
elements (the environment, objects, process, and people involved)
Service string-Scenario• Storyboard-that shows the pathway through
the service.• This should show the whole picture of the
service-from seeing the sign for the service to exit.
Saffer, 2010, p 156
Cognitive walkthroughs
• Follow the scenario using the flowcharts or paper prototype.
• Questions to ask:– Will the user realistically be trying to do this action?– Is the control for the action visible?– Is there a strong link between the control and the
action?– Is feedback appropriate?
From Low to High Fidelity prototypes• High fidelity prototypes are interactive prototypes
that represent various degrees of functionality
http://www.grameenfoundation.applab.org/challenges/what-innovative-mobile-money-mm-products-can-we-develop-that-are-suitable-for-poor-unbanked-customers-and-viable-for-our-commercial-partners/timeline.html
Nielson’s Heuristics ‘Rules of Thumb’
1. Visibility of system status2. Match between system and the real world3. User control and freedom4. Consistency and standards5. Error prevention6. Recognition rather than recall7. Flexibility and efficiency of use8. Aesthetic and minimalist design9. Help users recognize, diagnose, and recover from errors10. Help and documentation
11.
http://www.useit.com/papers/heuristic/heuristic_list.html
Heuristics Process
http://uxcentered.wordpress.com/2009/12/27/hello-world/
Heuristics• Experts • Set of criteria• Scenario where each step is tested against the
heuristic criteria.• Report
http://uxcentered.wordpress.com/2009/12/27/hello-world/
Final design
References• Bell, J. (1999) Doing your research project: A guide for first-time researchers in education and social science. (3rd Ed) Open University Press,
Buckingham, Philadelphia• Garrett, J. (2010) The elements of user experience: user-centred design for the web and beyond . New Riders Press, Barkley, CA • Goto, K. & Cotler, E. (2002). Web ReDesign. Workflow that Works. Indianapolis: New Riders.• Heim, S. (2008) The Resonant Interface HCI Foundations for Interaction Design, Pearson Addison-Wesley, New York• Norman, D. (2004). Emotional Design. New York: Basic Books. • Preece, J., Rogers, Y. & Sharp, H. (2002). Interaction Design: Beyond Human-Computer Interaction. New Jersey: John Wiley & Sons. • Saffer, D. (2010)Designing for interaction. (2nd ed) New Riders Press, Berkley, CA• Shneiderman, B. and Plaisant, C. (2005). Designing the User Interface: Strategies for Effective Human-Computer Interaction (Fourth Edition)
Pearson Addison Wesley, Boston, MA.• Torres, R.J. (2002). Practitioner's Handbook for User Interface Design in Development. New Jersey: Prentice Hall.• Van Duyne, D., Landay, J. & Hong, J (2003). The Design of Sites. New York. Addison- Wesley.
• www.useit.com/papers/heuristic/heuristic_list.html• www.cse.unr.edu/~dascalus/ID2011_11.ppt• http://uxcentered.wordpress.com/2009/12/27/hello-world/• http://www.grameenfoundation.applab.org• http://www.gliffy.com/examples/wireframes• http://www.nngroup.com/reports/prototyping/video_stills.html• http://www.userfocus.co.uk/articles/cogwalk.html