user interface evaluation usability inquiry methods zzj/usabilityhome.html

User Interface Evaluation

Usability Inquiry Methods

http://jthom.best.vwh.net/usability/http://www.cs.umd.edu/~zzj/UsabilityHome.html

Usability Inquiry Methods

Usability experts learn about the users’ likes, dislikes, needs, etc. of the system through: Observation Verbal questioning Written questioning

Widely used in practice.

Different methods have different costs, but in general, this is relatively cheap.

Usability Inquiry Methods Contextual Inquiry

Field Observation

Questionnaires

Interviews

Focus Groups

Logging Actual Use

Contextual Inquiry Contextual Inquiry is also known as proactive

field studies.

Applicable Stages: Requirements & Design

Personnel Usability Experts, approximately 1. Developers, 0. Users, 2.

Contextual Inquiry Usability Issues Covered

Effectiveness: No Efficiency: No Satisfaction: No

Quantitative Data is NOT collected.

Can NOT be conducted remotely.

Can be used on any system.

Contextual Inquiry What is it?

Before designing the system, the expert(s) visit the users’ workplace and question them.

This should occur before any design has been done.

Contextual Inquiry How can I do it?

Determine who your users are.

Go visit them where they work.

Contextual Inquiry How can I do it?

Talk to them about the system How do they currently do their job? How would you like to do your job? What do you like about the current system/method? What don’t you like about the current

system/method?

http://jthom.best.vwh.net/usability/context.htm http://www.cs.umd.edu/~zzj/FieldStu.htm

Field Observation Applicable Stages:

Test & Deployment

Personnel Usability Experts, approximately 1. Developers, 0. Users, 2.

Field Observation Usability Issues Covered

Effectiveness: Yes Efficiency: No Satisfaction: Yes

Quantitative Data is NOT collected.

Can NOT be conducted remotely.

Can be used on any system.

Field Observation What is it?

Usability experts observe users in the field using the system/product.

Field Observation How can I do it?

Go to the users’ workplace and simply observe.

Things to look for: What is the user’s mental model? Are the users using it the way you expect? You don’t want them to know you are evaluating

them.

http://www.cs.umd.edu/~zzj/FieldObs.htm

Questionnaires Applicable Stages:

Any stage of development (depending on the questions)

Personnel Usability Experts, approximately 1. Developers, 0. Users, 2.

Questionnaires Usability Issues Covered

Effectiveness: Yes Efficiency: No Satisfaction: Yes

Quantitative Data is NOT collected.

Can be conducted remotely.

Can be used on any system.

Questionnaires What is it?

Written lists of questions that you distribute to your users.

Questionnaires How can I do it?

Develop a list of questions on paper, web, email, etc. and give the questionnaire(s) to the users.

The users will answer the questions and return the questionnaires to you.

http://jthom.best.vwh.net/usability/question.htm http://www.acm.org/~perlman/question.html

Interviews Applicable Stages:

Any stage of development (depending on the questions)

Personnel Usability Experts, approximately 1. Developers, 0. Users, 1.

Interviews Usability Issues Covered

Effectiveness: Yes Efficiency: No Satisfaction: Yes

Quantitative Data is NOT collected.

Can be conducted remotely.

Can be used on any system.

Interviews What is it?

You interview users and ask them questions.

Interviews How can I do it?

Develop a list of questions for the users.

Meet with the users, individually.

Ask them the questions and log the responses Written and/or taped

Interviews How can I do it?

Interview Tips: Clearly define this is an interview. Ask open ended questions to get the user talking.

Yes-No questions are bad. Begin with less demanding topics and progress to

more difficult topics. Don’t ask questions to support your belief or

hypothesis. Do not answer your own questions. Do not agree or disagree … remain neutral.

Interviews How can I do it?

Probes: used to encourage the subjects to continue speaking, or to guide their response in a particular direction

Addition Probe Encourages more information or clarifies certain

responses from the test users. Either verbally or nonverbally the message is, "Go on, tell me more," or "Don't stop."

Interviews How can I do it?

Reflecting Probe Uses a nondirective technique, encourages the test

user to give more detailed information. The interviewer can reformulate the question or synthesize the previous response as a proposition.

Directive Probe Specifies the direction in which a continuation of

the reply should follow without suggesting any particular content. A directive probe may take the form of "Why is the (the case)?"

Interviews How can I do it?

Defining Probe Requires the subject to explain the meaning of a

particular term or concept.

http://jthom.best.vwh.net/usability/surveys.htm

http://www.cs.umd.edu/~zzj/Interview.htm

Focus Groups Applicable Stages:

Test & Deployment

Personnel Usability Experts, approximately 1. Developers, 0. Users, 6.

Focus Groups Usability Issues Covered

Effectiveness: Yes Efficiency: No Satisfaction: Yes

Quantitative Data is NOT collected.

Can be conducted remotely with email, bulletin boards, conference calls, etc.

Can be used on any system.

Focus Groups What is it?

A group of users are gathered to talk about the system.

The expert acts as the moderator.

Should conduct more than 1 focus group.

Focus Groups How can I do it?

Bring a group of users together and begin.

Collect data

http://www.cs.umd.edu/~zzj/FocusGro.htm

Logging Actual Use Applicable Stages:

Test & Deployment

Personnel Usability Experts, approximately 1. Developers, 0. Users, 6.

Logging Actual Use Usability Issues Covered

Effectiveness: Yes Efficiency: Yes Satisfaction: Maybe

Quantitative Data is NOT collected, in most cases.

Can be conducted remotely.

Can be used on most systems.

Logging Actual Use What is it?

The computer automatically collect usage data.

You could ask the user to log their usage, but that’s not practical.

Logging Actual Use How can I do it?

Usability Logger http://www.usabletools.com/

Automatic capture of keyboard, mouse, etc.

VideoCam and other products.

Logging Actual Use Facts On Logging Actual Use

You know exactly what the user is doing. You don’t know why, but you do know what, when,

where. You don’t know how the user feels.

http://www.cs.umd.edu/~zzj/Logging.htm

Usability Inquiry Methodsmost useful

Contextual Inquiry

Field Observation

Questionnaires

Interviews

Focus Groups (least useful)

Logging Actual Use