rationalising and designing methodologies for new media research gosia kwiatkowska gosia@uel.ac.uk

Rationalising and Designing Methodologies for New Media

Research

Gosia Kwiatkowskagosia@uel.ac.uk

Designing New Media

Choosing your research method

Quantitative or Qualitative Method or both?

Quantitative Methods

– Designed to ensure objectivity, reliability and ability to generalise

– Test predetermined hypothesis – explanations– Who, how much, how many?– Closed questions– Short answers– Averages, percentages, ranges, means, frequencies– Can generate accurate and precise data – Can test statistical relationship between variable– Can prove whether or not a particular problem exists– Can identify specific characteristics of a population

http://documents.wfp.org/stellent/groups/public/documents/ko/mekb_module_13.pdf

Quantitative Methods

• Strengths– Robust – Objective– Verifiable

• Weaknesses– Out of context – human behaviour, real world

settings are not considered – Any variables left out of data collection are not

used in analysis

Qualitative Method?– User perspectives – Questions: how and why?– Captures perceptions, judgements, meanings,

processes and reasons– Open ended questions, checklist of topics– Hypothesis and follow up questions are generated

during data collection and analysis– Subjective – not easy to objectively verify– Interviews allow to probe – Provides deeper insight – Broader understanding and explanations, views

http://documents.wfp.org/stellent/groups/public/documents/ko/mekb_module_13.pdf

Qualitative Method?• Strengths

– Participatory– Rich, detailed data – Considers users perspectives and the context for their

behaviours • Weaknesses

– Hard work– Time consuming– Smaller sample of users– Not easily verifiable– Not easy to group your responses/categorised

Quantitative or Qualitative or Both?

• Quantitative methods can highlight an issues that could then be studies in depth using qualitative methods.

• Qualitative methods might be used at the beginning of a study to help the researcher to decide what closed questions could be used in the bigger quantitative survey or questionnaire.

Design phases

• Phase 1 - User analysis– Questionnaires/ethnography/interviews

• Phase 2 - Conceptual design– Cards sorts/cognitive walkthrough – flowcharts

• Phase 3 - Testing and implementation– User testing of prototypes– Final design

Phase One- User Analysis

http://www.grameenfoundation.applab.org/challenges/what-innovative-mobile-money-mm-products-can-we-develop-that-are-suitable-for-poor-unbanked-customers-and-viable-for-our-commercial-partners/timeline.html

Tools

• Surveys: type of questions?• Interviews: interview questions? • Focus groups: purpose and objectives,

discussion notes, who, why, where? What did you find out? Which quotes, comments will you use to support your findings?

• Observations: observation checklist• Case studies: use triangulation

Checklist for designing and conducting interviews, focus groups, surveys

• Decide what you need to know.• State why you need to know this information.• What is the best way of obtaining this

information.• Outline your questions.• Check wording of each question. • Decide on the tool for data collection and the

question type.• Refine the questions and order them.• Consider how questions will be analysed.

Checklist for designing and conducting interviews, focus groups, surveys cnt

• Write instruction that might be included in your surveys.• Consider layout and appearance.• Decide on your sample, are they representative.• Prepare a schedule. • Pilot.• Revise, if necessary.• Do your best to avoid bias. • Select who to interview.• Ensure official channels have been cleared. Information

sheets, consents etc• Introduce yourself and explain the purpose of the research

Checklist for designing and conducting interviews, focus groups, surveys cnt

Designing your questionnaires

• Questions types– List– Category– Ranking– Scale– Quantity– Grid– Verbal

List

Summary sheetQuestion 13: What do you use the internet for?

Finding interesting information

Finding pictures

Finding and watching videos

Finding and listening to music

Online learning

Shopping Online gaming

Other

||||| || | ||| || |

|||

17 18 12 21 18 27 26 3

Category questions

• How old are you? ______

What will you do with the responses? Depends on what you want to know?e.g. Average age of your users? If so, what sort

of average or measure of central tendency will suit your purpose: the arithmetic means, the median, or the mode?

The arithmetic mean

• Arithmetic mean = sum or each value/total number of values.

• 26 participants• Ages: 21,22,22,25,25,25,25,25,25,26,26, 26,

27, 28, 29, 30,30,32,32,33,34,35,35,35,35,35• Add ages together = 743• Divide 743 by 26 • The mean = 28.57

Mean

• Group A Ages: 21,22,22,25,25,25,25,25,25,26,26, 26,

27, 28, 29, 30,30,32,32,33,34,35,35,35,35,35• Group B Ages: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21,

21, 22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65,

• The same mean but clear difference between the dispersion of results

The median• Allows us to find the middle valueIf we have an odd number of values If we have an even number of values: take the average of

the middle valuesGroup AAges: 21,22,22,25,25,25,25,25,25,26,26, 26,27,28, 29,

30,30,32,32,33,34,35,35,35,35,35Median=(27+28)/2 = 27.5Group BAges: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21,

22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65Median =22.5You decide which one gives more realistic picture.

The mode

• The most frequently occurring valueGroup AAges: 21,22,22,25,25,25,25,25,25,26,26, 26,27,28,

29, 30,30,32,32,33,34,35,35,35,35,35Modal score = 25Group BAges: 18,18, 18, 19, 19,19, 19, 20, 20, 20, 21, 21,

22,23,24,25,26,29,30,39,39, 43, 45, 49, 52,65Modal score = 19

Range

Range the difference between the highest and the lowest values measured

• Group A 21-35 range = 35-21 = 14 • Group B 18-65 range = 65-18 = 47

Problem? What about the middle numbers?

Interquartile range• Interquartile range indicates the extent to which data

vary.• Derived from the median, the highest and the lowest

quarter of the measures are omitted and the interquartile range of the middle 50 % of values is quoted

Group A 21,22,22,25,25,25,25,25,25,26,26, 26,27,median point28, 29, 30,30,32,32,33,34,35,35,35,35,35Interquartile range is 25-32 or 7 years

Alternative question

• How old are you?18-29 30-39 40-49 50-59 60+

You can still calculate the arithmetic means by using the mid point of each class interval.

Scales

Verbal (open) question

Recording your data

• Data must be recorded, analysed and interpreted.

• Identify categories.• Prepare summary sheets.• Experiment with different ways of presenting

findings.• Remember all data require interpretation.

Presenting your data and analysis

Phase Two-Conceptual design

Making sense of data Organise data Aggregation (combine/collapse redundant pieces of data) Juxtaposing related pieces of data (putting together) Naming the resulting data clusters Juxtaposing unrelated pieces of data

Exploring alternative solutions Choosing a design

(Safer, 2010, p.95)

Conceptual design-Tools

– Brainstorming– Card sort– Personas– Scenarios– Flowcharts– Use cases

Brainstorming

Card Sort

• Step 1 – separate all information • Step 2 – organise the cards into related areas• Step 3 – label your groups• What if one topic fits into more than one

category?• Test• This will provide basis for your information

architecture, the structure and the design.

Card Sort

Advantages• Can be done before design• Quick and easy• Good way of finding out

underlying structures for your design

• Good way of finding out how the end user organises information

Disadvantages• Difficult to navigate when

you have lots of categories• Limited to the topics that

you put on the cards• Card sort outcomes suggest

structures

Personas

http://geoinformatics.fsv.cvut.cz/gwiki/Developing_web_map_application_based_on_user_centered_design

From Personas to Cognitive Walkthroughs

• Scenarios: typical task, goal, conditions at the beginning of the task, activities in which the persona will engage, the outcomes of these activities.

Scenario

http://csweb.cs.bgsu.edu/maner/domains/Scenario.htm

Flowcharts

Information architecture• Organise information• Show navigational structure• Efficiency• Effectiveness

It should correspond to:• Content• User needs • Our objectives

Garrett, 2010, p94

Use cases

Phase Three-Testing and Implementation

The physical design:– Look – Components– Layout

Tools:– Low-fidelity prototypes– Evaluations– Wireframes– High-fidelity prototypes

Low-fidelity prototyping

• Nielsen’s two types of prototypes – Horizontal– Vertical

http://www.useit.com/papers/guerrilla_hci.html

Low-fidelity prototypes:

• Easy and cheap to make• Flexible-can be changed and re-arranged• Complete enough to give useful feedback about specific

design question/s

http://www.nngroup.com/reports/prototyping/video_stills.html

Nielsen Norman Group

Things to consider in order to define the scope of the prototype and the focus:

• What feedback do you need at this point in the design process?• How much of the design should you prototype?• Should you cover all of the areas but without great detail (breadth vs.

depth)?• Should you cover one area in great detail?

Nielsen Norman Group

Evaluation

Begin evaluations early in the design process.

• Evaluation is an integral part of the development process and can take the form of an informal walkthrough or a more structured heuristic evaluation.

• Formal usability testing can begin once a prototype has been developed.

Wireframe-blueprint of a product

Wireframes cnt

• Wireframes help to create template layouts that can be used to impose a consistent structure throughout the interface

http://www.gliffy.com/examples/wireframes/

Service blueprint

• Similar like wireframe for digital product, service blueprint presents service moments and service string. E.g. car wash

• Brainstorm possible service moments• Sketch/photograph/single storyboard frame

each moment• How each service moment affects service

elements (the environment, objects, process, and people involved)

Service string-Scenario• Storyboard-that shows the pathway through

the service.• This should show the whole picture of the

service-from seeing the sign for the service to exit.

Saffer, 2010, p 156

Cognitive walkthroughs

• Follow the scenario using the flowcharts or paper prototype.

• Questions to ask:– Will the user realistically be trying to do this action?– Is the control for the action visible?– Is there a strong link between the control and the

action?– Is feedback appropriate?

From Low to High Fidelity prototypes• High fidelity prototypes are interactive prototypes

that represent various degrees of functionality

http://www.grameenfoundation.applab.org/challenges/what-innovative-mobile-money-mm-products-can-we-develop-that-are-suitable-for-poor-unbanked-customers-and-viable-for-our-commercial-partners/timeline.html

Nielson’s Heuristics ‘Rules of Thumb’

1. Visibility of system status2. Match between system and the real world3. User control and freedom4. Consistency and standards5. Error prevention6. Recognition rather than recall7. Flexibility and efficiency of use8. Aesthetic and minimalist design9. Help users recognize, diagnose, and recover from errors10. Help and documentation

11.

http://www.useit.com/papers/heuristic/heuristic_list.html

Heuristics Process

http://uxcentered.wordpress.com/2009/12/27/hello-world/

Heuristics• Experts • Set of criteria• Scenario where each step is tested against the

heuristic criteria.• Report

http://uxcentered.wordpress.com/2009/12/27/hello-world/

Final design

References• Bell, J. (1999) Doing your research project: A guide for first-time researchers in education and social science. (3rd Ed) Open University Press,

Buckingham, Philadelphia• Garrett, J. (2010) The elements of user experience: user-centred design for the web and beyond . New Riders Press, Barkley, CA • Goto, K. & Cotler, E. (2002). Web ReDesign. Workflow that Works. Indianapolis: New Riders.• Heim, S. (2008) The Resonant Interface HCI Foundations for Interaction Design, Pearson Addison-Wesley, New York• Norman, D. (2004). Emotional Design. New York: Basic Books. • Preece, J., Rogers, Y. & Sharp, H. (2002). Interaction Design: Beyond Human-Computer Interaction. New Jersey: John Wiley & Sons. • Saffer, D. (2010)Designing for interaction. (2nd ed) New Riders Press, Berkley, CA• Shneiderman, B. and Plaisant, C. (2005). Designing the User Interface: Strategies for Effective Human-Computer Interaction (Fourth Edition)

Pearson Addison Wesley, Boston, MA.• Torres, R.J. (2002). Practitioner's Handbook for User Interface Design in Development. New Jersey: Prentice Hall.• Van Duyne, D., Landay, J. & Hong, J (2003). The Design of Sites. New York. Addison- Wesley.

• www.useit.com/papers/heuristic/heuristic_list.html• www.cse.unr.edu/~dascalus/ID2011_11.ppt• http://uxcentered.wordpress.com/2009/12/27/hello-world/• http://www.grameenfoundation.applab.org• http://www.gliffy.com/examples/wireframes• http://www.nngroup.com/reports/prototyping/video_stills.html• http://www.userfocus.co.uk/articles/cogwalk.html