observing users presenter : jungmin shin & jaeyoung lee

Observing usersObserving users

Presenter : JungMin shin & JaeYoung LeePresenter : JungMin shin & JaeYoung Lee

ContentsContents

Goals, questions, paradigmsGoals, questions, paradigms How to observeHow to observe Data CollectionData Collection Indirect Observation: tracking user’s Indirect Observation: tracking user’s

activitiesactivities Analyzing, interpreting, and Analyzing, interpreting, and

presenting data presenting data Field studiesField studies

Goals, questions, Goals, questions, paradigmsparadigms

What and when to observeWhat and when to observe Approached to observationApproached to observation


Goal and question provide a focus for observation,Goal and question provide a focus for observation,

DECIDE frameworkDECIDE framework..

Having goal helps to guide the observation because Having goal helps to guide the observation because

there is always so much going on.there is always so much going on.

What and when to observeWhat and when to observe On developing product On developing product

starting time : helps understand starting time : helps understand user’s needsuser’s needs..

ending time : examine to satisfy ending time : examine to satisfy user’suser’s needsneeds..


Role of evaluator Role of evaluator Onlookers, participant observers, ethnographersOnlookers, participant observers, ethnographers

Table 12.1 Type of observationTable 12.1 Type of observation

ObservationObservation Controlled Controlled environment environment (i.e., lab-like)(i.e., lab-like)

Field Field environmentenvironment

(i.e., natural)(i.e., natural)

Outsider Outsider looking onlooking on

““Quick and dirty” Quick and dirty”

In usability testingIn usability testing““Quick and dirty”Quick and dirty”

In field studiesIn field studies

InsiderInsider (Not applicable)(Not applicable) Participant Participant observation observation (e.g., (e.g., ethnography)ethnography)


Approaches to observationApproaches to observation1. “Quick and dirty” observation1. “Quick and dirty” observation

Watching and talking to users in a casual way to Watching and talking to users in a casual way to get immediate feedback. get immediate feedback.

2. Observation in usability testing2. Observation in usability testingVideo and interacting logs capture everything.Video and interacting logs capture everything.Can watch through a one-way mirror or via TV Can watch through a one-way mirror or via TV screen. screen. Observational data is used to see and analyze Observational data is used to see and analyze

for for user’s action.user’s action.

Observation in usability testing SystemObservation in usability testing System



3. Observation in field studies3. Observation in field studiesObservers may be anywhere along the outsider –Observers may be anywhere along the outsider –insider spectrum.insider spectrum.

Colin Robson(1993) – participating levelColin Robson(1993) – participating levelcomplete participantscomplete participantsmore marginal participantsmore marginal participants

observers who also participateobservers who also participatepeople who observepeople who observe from the outside and do not from the outside and do not participate.participate.

The goal is to cause as little disruption as The goal is to cause as little disruption as possible.possible.


3. Observation in field studies (Continue)3. Observation in field studies (Continue) ex)ex)

A study of the time spent by boy and girl A study of the time spent by boy and girl using technology in the classroom.using technology in the classroom.-> observe and note at the back of room.-> observe and note at the back of room.

A study of understanding how the A study of understanding how the computer integrates with other artifacts computer integrates with other artifacts and social interacting.and social interacting.-> might take of insider perspective. -> might take of insider perspective.


3. Observation in field studies (Continue)3. Observation in field studies (Continue)

participant observation evaluatorparticipant observation evaluatorparticipate with users in order to participate with users in order to

learn learn what they do and how and why what they do and how and why they do it.they do it.

fully participant observerfully participant observera member of group.a member of group.being present being present to share experienceto share experience

and and learning the social conventions of the learning the social conventions of the group.group.


Determining goals , exploring questions Determining goals , exploring questions Choosing Choosing

techniques are necessary steps techniques are necessary steps in the in the DECIDE DECIDE

framework.framework.

In controlled environmentsIn controlled environments In the field environmentsIn the field environments Participant observation and Participant observation and

ethnographyethnography

How to observeHow to observe

In controlled environmentsIn controlled environmentsCollect Collect and then and then Make sense of theMake sense of the datadata

whilewhile

watching users in controlled environment.watching users in controlled environment.

Practical issues in controlled environment Practical issues in controlled environment

- where users will be located - where users will be located

- equipment test- equipment test

- An informed consent form- An informed consent form


In controlled environmentsIn controlled environmentsthe observer doesn’t know what users are the observer doesn’t know what users are thinking, and can only guess from what she sees.thinking, and can only guess from what she sees.

Think-aloud techniqueThink-aloud technique by Erikson and Simon by Erikson and SimonRequire people to say out loud everything that they are Require people to say out loud everything that they are

thinking and thinking and trying to do.trying to do.ex) I’m typing in ex) I’m typing in http://ics.uci.eduhttp://ics.uci.edu (type) (type) Now I press the enter key, right? (presses enter key)Now I press the enter key, right? (presses enter key)

But if users meet some errors, they could be silent again...But if users meet some errors, they could be silent again...

Sol) Evaluator reminds him think out loud, but that would be intrusive.

-> two people work together so that they talk to each other.


In the field environmentsIn the field environmentsEvents in the field can be complex and rapidly Events in the field can be complex and rapidly changing.changing.

FrameworkFramework help observers to keep their goals and help observers to keep their goals and

questions in questions in sight.sight.

Collin Robson(1993)Collin Robson(1993)Space, Actors, Activities, Objects, Acts, Space, Actors, Activities, Objects, Acts,

Events, Goals, Events, Goals, FeelingsFeelings


In the field environmentsIn the field environments Checklist of things to plan before going into fieldChecklist of things to plan before going into field

1. State the initial study 1. State the initial study goal and questiongoal and question2. Select the 2. Select the frameworkframework3. Decide 3. Decide how to recordhow to record events events4. Be prepared to go through your notes and other records4. Be prepared to go through your notes and other records5. Make and review your notes5. Make and review your notes6. Be prepared refocus your study6. Be prepared refocus your study7. Think about how you will get the acceptance and trust of 7. Think about how you will get the acceptance and trust of those you observethose you observe8. Think about how to handle sensitive issues (privacy)8. Think about how to handle sensitive issues (privacy)9. Consider working as a team9. Consider working as a team10. Consider checking your notes with an informant or 10. Consider checking your notes with an informant or

membersmembers of the groupof the group11. Plan to look at the situation from different perspectives11. Plan to look at the situation from different perspectives


Participant observation and Participant observation and ethnographyethnography

Checklist for doing ethnographyChecklist for doing ethnography Identifying a problems or goalIdentifying a problems or goal and then ask good and then ask good

questions to be answered by the studyquestions to be answered by the study The most important part of fieldwork just The most important part of fieldwork just being being

there to observe, ask questions, and record.there to observe, ask questions, and record. CollectCollect a variety of data (Interview, retrospectives a variety of data (Interview, retrospectives

interview)interview) Be prepared Be prepared to moveto move backwards and forwards backwards and forwards

between the broad picture and specific questionsbetween the broad picture and specific questions Analyze Analyze the data using a holistic approach -> the data using a holistic approach ->

contextualized, usually iterativecontextualized, usually iterative


Participant observation and Participant observation and ethnographyethnography

<Dilemma><Dilemma>When have I observed enough?When have I observed enough?

schedules often dictate schedules often dictate when your study endwhen your study end..

stop stop when you stop learning new thingswhen you stop learning new things

when you start when you start to see similar patternto see similar pattern of behavior of behavior being repeated.being repeated.

when you have when you have listened to all the main stake listened to all the main stake holder group and understood their holder group and understood their perspectivesperspectives..


How to observeHow to observe Participant observation and ethnographyParticipant observation and ethnography

How can I adapt ethnography to fit the How can I adapt ethnography to fit the development development

process?process?Ann Rosa ProcedureAnn Rosa Procedure

PreparingPreparing (Understand/Familiarize/Set Goal (Understand/Familiarize/Set Goal /Gain /Gain

permission) permission) Field studyField study (Establish/Observe and interview/ (Establish/Observe and interview/

Follow any lead/ Record)Follow any lead/ Record)AnalysisAnalysis (Compile/Quantify/Reduce and (Compile/Quantify/Reduce and

interpret/ interpret/ Refine the goals) Refine the goals) ReportingReporting (Consider/Prepare) (Consider/Prepare)

Data collectionData collection Notes plus still cameraNotes plus still camera Audio recording plus still cameraAudio recording plus still camera VideoVideo

Data collectionData collection Notes plus still cameraNotes plus still camera

being difficult and tiring to write and observe at the being difficult and tiring to write and observe at the same timesame timebored, the limitation of writing speed bored, the limitation of writing speed

(sol) working with another person(sol) working with another person(disadvantage) have to be transcribed (disadvantage) have to be transcribed

Audio recording plus still cameraAudio recording plus still cameraUseful alternative to note taking and is Useful alternative to note taking and is less less

intrusive than intrusive than Video.Video.Drawback:Drawback: Lack of visual record, transcribing the Lack of visual record, transcribing the

datadataSo,So, evaluators use the recording evaluators use the recording to remindto remind them them

about about important details and as a source of anecdotes for important details and as a source of anecdotes for

reports.reports.

Data collectionData collection VideoVideo

AdvantageAdvantage Capturing both Audio and Video DataCapturing both Audio and Video Data

DisadvantageDisadvantageIntrusive.Intrusive.Attention becomes focused on what is Attention becomes focused on what is seen through the lensseen through the lens..Time-consuming for analyzing video data.Time-consuming for analyzing video data. ex) 1 hour video recording : over 100 hours ex) 1 hour video recording : over 100 hours of of

analysis timeanalysis time

Indirect Observation: Indirect Observation: tracking user’s activitiestracking user’s activities

DiariesDiaries Interaction logsInteraction logs


When ? Impossible to observe directly When ? Impossible to observe directly

DiariesDiariesA recordA record of what users did, when did it, what thought of what users did, when did it, what thought

about their about their interactions with the Technology.interactions with the Technology.

(Advantage) (Advantage) When User scattered and unreachable person.When User scattered and unreachable person.Inexpensive - no equipment or expertise.Inexpensive - no equipment or expertise.Suitable for long term studies.Suitable for long term studies.Template and Enable the data to go straight into database. Template and Enable the data to go straight into database. Ex) open-ended online questionnairesEx) open-ended online questionnaires

(Disadvantage)(Disadvantage)Rely on participant (reliable/remembering).Rely on participant (reliable/remembering).Participant remember better or worse that they really Participant remember better or worse that they really

were.were.

Interaction logsInteraction logskey presses, mouse or other device movement are recorded usually key presses, mouse or other device movement are recorded usually synchronized with A/V log.synchronized with A/V log.Has time-stamped (to calculate how long)Has time-stamped (to calculate how long)Explicit counter / Recording number of visitorsExplicit counter / Recording number of visitors

EX) In study of interact art museum (Researchers at USC)EX) In study of interact art museum (Researchers at USC)Server log (7 months/ analysis tool : webtrends)Server log (7 months/ analysis tool : webtrends)when/what /how long/what browser/what country…when/what /how long/what browser/what country…

(Advantage)(Advantage)Unobtrusive (but this also raises ethical concerns)Unobtrusive (but this also raises ethical concerns)Large volumes of data.Large volumes of data.


Analyzing, interpreting, Analyzing, interpreting, and presenting dataand presenting data

The first things to do is to “eyeball”. The first things to do is to “eyeball”. Proceed to analyze it according to the goals andProceed to analyze it according to the goals andquestions.questions.

Qualitative analysis to tell a storyQualitative analysis to tell a story Qualitative analysis for Qualitative analysis for

categorizationcategorization Quantitative data analysisQuantitative data analysis Feeding the findings back into Feeding the findings back into

designdesign

Qualitative analysis to tell a story Qualitative analysis to tell a story Power of analyzing descriptive data -> being Power of analyzing descriptive data -> being

able able

to tell a convincing storyto tell a convincing story

Interview with Sara Bly Interview with Sara Bly (User-Centered Design (User-Centered Design Consultant) Consultant)

At the end of each observation periodAt the end of each observation period

Review their dataReview their data

Discuss what they observedDiscuss what they observed

Construct a story from their dataConstruct a story from their data

Analyzing, interpreting, Analyzing, interpreting, and presenting data and presenting data

Qualitative analysis to tell a storyQualitative analysis to tell a story To summarize about Interview with Sara Bly.To summarize about Interview with Sara Bly.

Review the data.Review the data.Record the themes.Record the themes.Record the date and time of each data analysis Record the date and time of each data analysis Session.Session.As themes emerge.As themes emerge.Iterate this process (until your story faithfully Iterate this process (until your story faithfully represents).represents).Report your findings to the development team.Report your findings to the development team.


Qualitative analysis to tell a storyQualitative analysis to tell a story

Analyzing and reporting ethnographic dataAnalyzing and reporting ethnographic data- ethnographers reconstruct knowledge to produ- ethnographers reconstruct knowledge to produ

ce detailed description known as rich or thick descriptice detailed description known as rich or thick description.on.

Main activities (Fetterman-1998)Main activities (Fetterman-1998) Look for key eventsLook for key events Look for patterns of behavior in various situations and among diLook for patterns of behavior in various situations and among di

fferent playersfferent players Compare sourcesCompare sources Report your findings in convincing and honest wayReport your findings in convincing and honest way

S/W tools : NUDIST and EthnographS/W tools : NUDIST and Ethnograph


Qualitative analysis for categorizationQualitative analysis for categorizationLooking for incident or patternsLooking for incident or patterns

Common strategy is to look for critical incidentsCommon strategy is to look for critical incidents

ex) Jurgen Koenemann-Belliveau el al(1994)ex) Jurgen Koenemann-Belliveau el al(1994)used this form of critical incident analysis to examinused this form of critical incident analysis to examin

e e breakdown or problems in achieving a programming tbreakdown or problems in achieving a programming t

ask ask and also to identify possible threats of incidents.and also to identify possible threats of incidents.

Using the theory helped the evaluators to focus in relevant Using the theory helped the evaluators to focus in relevant Incidents.Incidents.


Qualitative analysis for categorizationQualitative analysis for categorizationanalyzing data into categoriesanalyzing data into categoriescontents analysis provides another fine grain way of contents analysis provides another fine grain way of analyzing video data.analyzing video data.determined by the evaluation question.determined by the evaluation question.must also be reliable so that the analysis can be replicated.must also be reliable so that the analysis can be replicated.

EX)EX)training second person and then analyze the same data training second person and then analyze the same data sample. sample. if there are large discrepancy between each other. -> what?if there are large discrepancy between each other. -> what?

Inter-research reliability rating Inter-research reliability rating -> the percentage of agreement between the two -> the percentage of agreement between the two

researchers.researchers.


Analyzing discourseAnalyzing discourseDialogDialogStrongly interpretive, pays great attention to Strongly interpretive, pays great attention to

context.context.

Language is viewed as a constructive tool and discourse Language is viewed as a constructive tool and discourse analysis provides a way of focusing upon how people use analysis provides a way of focusing upon how people use language to construct versions of their worlds (Fiske,1994)language to construct versions of their worlds (Fiske,1994)

Small changes in wording can change meaning, as the Small changes in wording can change meaning, as the following excerpts indicate (Coyle, 1995)following excerpts indicate (Coyle, 1995)

(According to Coyle,)(According to Coyle,) discourse analysis is what you do discourse analysis is what you do when when

you are saying that you are doing discourse analysis…you are saying that you are doing discourse analysis…


Quantitative data analysisQuantitative data analysisIn case of errors or unusual behavior ->In case of errors or unusual behavior ->

marks the video and records a brief marks the video and records a brief remark. remark.

To use this annotated recording To use this annotated recording

1. Calculate performance time 1. Calculate performance time

2. further analyzed using simple statistics 2. further analyzed using simple statistics such as means, standard deviations, T-such as means, standard deviations, T-tests, etc.tests, etc.

Analyzing, interpreting, Analyzing, interpreting, andand

presenting data presenting data

Feeding the findings back into designFeeding the findings back into designThe results can be The results can be reported to the design teamreported to the design team..

Written reports with a overview and detailed Written reports with a overview and detailed

content list.content list.

Quantitative data -> Quantitative data -> its value depends on the type its value depends on the type

of study and its goals.of study and its goals.

Often both quantitative data analysis are useful Often both quantitative data analysis are useful

because of they because of they provide alternative perspectivesprovide alternative perspectives. .


Field StudyField Study

Field StudyField Study

A broad range of data gathering A broad range of data gathering techniques at the techniques at the user’s locationuser’s location

There is There is no one best wayno one best way to conduct to conduct a field study. a field study. It depends on the goals It depends on the goals of your study and your access to of your study and your access to users.users.

The Goal of a Field StudyThe Goal of a Field Study

Field studies are Field studies are excellentexcellent for for collecting collecting rich, detailed datarich, detailed data and and obtaining a obtaining a holistic view of the process or domainholistic view of the process or domain..

Field studies are Field studies are not a good techniquenot a good technique if if you you want to collect quantitative data want to collect quantitative data from large samples.from large samples.

The goal of a field study is The goal of a field study is to make the to make the implicit aspects and processes of the implicit aspects and processes of the user environment explicit.user environment explicit.

Field Study MethodsField Study Methods

Techniques are divided into three Techniques are divided into three categories:categories: Observation onlyObservation only Interacting with the userInteracting with the user Method supplementsMethod supplements

TheThe Goal Goal of the Field Study Methodsof the Field Study Methods To To observe users and collect informationobserve users and collect information

about their tasks and the context in about their tasks and the context in which they are done.which they are done.

Field Study MethodsField Study Methods Observation OnlyObservation Only

Pure ObservationPure Observation Deep Hanging-OutDeep Hanging-Out

Interacting with the userInteracting with the user Contextual InquiryContextual Inquiry Process analysisProcess analysis Condensed ethnographic interviewCondensed ethnographic interview Discount User Observation (DUO)Discount User Observation (DUO)

Method supplementsMethod supplements Artifact walkthroughsArtifact walkthroughs Incident diariesIncident diaries Observing while you are not presentObserving while you are not present


Observation OnlyObservation Only Pure ObservationPure Observation Deep Hanging-OutDeep Hanging-Out

Observation only - Observation only - Pure Pure ObservationObservation

SynopsisSynopsis When you are When you are unable or don’t wish to interact unable or don’t wish to interact

with the userwith the user, you simply observe from a distance., you simply observe from a distance. AdvantagesAdvantages

FlexibleFlexible Low resourcesLow resources

Level of effortLevel of effort Minimal, Place your self in a good vantage point and Minimal, Place your self in a good vantage point and

observe as many user/sites/tasks as you feel appropriateobserve as many user/sites/tasks as you feel appropriate You continue to conduct observations until you feel you You continue to conduct observations until you feel you

have a good understanding of the domain of areas of have a good understanding of the domain of areas of focus.focus.

Observation only - Observation only - Pure Pure ObservationObservation

Valuable in situations where you cannot Valuable in situations where you cannot interact with the end users. interact with the end users.

Users may or may not know they are being Users may or may not know they are being studied. studied.

Do not interact with the participantDo not interact with the participant Do not distribute surveys, interview the Do not distribute surveys, interview the

user, or ask for artifacts from the user. user, or ask for artifacts from the user. It is essential to have a good sampling planIt is essential to have a good sampling plan. .

The sampling The sampling plan should include specific plan should include specific information such as days/times you information such as days/times you anticipate key events (e.g., the day before anticipate key events (e.g., the day before Thanksgiving, or bad weather at an airport), Thanksgiving, or bad weather at an airport), as well as “normal” daysas well as “normal” days..


Observation OnlyObservation Only Pure ObservationPure Observation Deep Hanging-OutDeep Hanging-Out

Observation only – Observation only – Deep Deep Hanging outHanging out

SynopsisSynopsis This method is similar to pure observation but This method is similar to pure observation but provides provides

more structure by suggesting focus areas and thingsmore structure by suggesting focus areas and things to to observe. observe.

AdvantagesAdvantages It has It has more structured than pure observationmore structured than pure observation so you can so you can

do a more detailed level of data analysis and compare do a more detailed level of data analysis and compare data collected across multiple sites.data collected across multiple sites.

Level of effortLevel of effort Because there is more structure, it takes more effort Because there is more structure, it takes more effort

than pure observation.than pure observation. You are “on” at all times, which can be tiring.You are “on” at all times, which can be tiring. It is also valuable to become a user yourself (if possible) It is also valuable to become a user yourself (if possible)

and collect artifacts along the way.and collect artifacts along the way.

Observation only – Observation only – Deep Deep Hanging outHanging out

A more structured form of pure observation.A more structured form of pure observation. Involves significant amounts of observation Involves significant amounts of observation

along with involving yourself in the processalong with involving yourself in the process Method of Deep Hanging-Out includesMethod of Deep Hanging-Out includes

Structured observationStructured observation Collection of artifactsCollection of artifacts Becoming a user yourselfBecoming a user yourself

However, However, you do notyou do not interview participants, interview participants, distribute surveys, or present design ideas distribute surveys, or present design ideas for feed back.for feed back.

Interacting with the user - Interacting with the user - Contextual InquiryContextual Inquiry

SynopsisSynopsis Interview, apprentice with, and interpret the Interview, apprentice with, and interpret the

resulting data with usersresulting data with users AdvantagesAdvantages

Contextual Inquiry is more Contextual Inquiry is more focused and context focused and context dependentdependent than the other methods than the other methods

At the end, you walk away with actionable itemsAt the end, you walk away with actionable items Level of effortLevel of effort

The effort level is The effort level is higher than for observation-higher than for observation-only techniquesonly techniques. You must develop an . You must develop an observation guide, observe users, apprentice observation guide, observe users, apprentice with them, and discuss your observations with with them, and discuss your observations with themthem


Four main parts to Contextual Four main parts to Contextual Inquiry:Inquiry: Context Context PartnershipPartnership InterpretationInterpretation FocusFocus


Four main parts to Contextual Four main parts to Contextual Inquiry:Inquiry:

ContextContext : You must : You must go to the user’s go to the user’s environmentenvironment in order to understand the in order to understand the context of his/her actions. Contextual context of his/her actions. Contextual Inquiry assumes that Inquiry assumes that observation alone observation alone or out-of-context interviews are or out-of-context interviews are insufficientinsufficient


Four main parts to Contextual Inquiry:Four main parts to Contextual Inquiry:

PartnershipPartnership : To better understand the : To better understand the user, tasks, and environment, you shoulduser, tasks, and environment, you should develop a master-prentice relationship with develop a master-prentice relationship with the participantthe participant. Immerse your self in the . Immerse your self in the participant’s work and do as he/she does. participant’s work and do as he/she does. Obviously, this is not possible with many Obviously, this is not possible with many jobsjobs (e.g., surgeon, fighter pilot) (e.g., surgeon, fighter pilot)



InterpretationInterpretation : : Observations must be Observations must be interpreted with the participantinterpreted with the participant in order in order to be used later. to be used later. Verify that your Verify that your assumptions and conclusions are assumptions and conclusions are correctcorrect..



FocusFocus : : Develop an observation guideDevelop an observation guide to keep you focused on the subject of to keep you focused on the subject of interest/inquiry.interest/inquiry.

Interacting with the user – Interacting with the user – Process Process AnalysisAnalysis

SynopsisSynopsis Capture the task sequence for a processCapture the task sequence for a process

that may span several daysthat may span several days AdvantagesAdvantages

Because it is more focused than Contextual Because it is more focused than Contextual Inquiry, Inquiry, it is also much fasterit is also much faster

Level of effortLevel of effort You need to You need to stay focused on the processstay focused on the process at at

hand to help users walk you through the hand to help users walk you through the process of interest. It may span just several process of interest. It may span just several days.days.


A process analysis is similar to Contextual A process analysis is similar to Contextual Inquiry. However, you enter with Inquiry. However, you enter with a series of a series of questionsquestions and you and you do not necessarily do not necessarily apprentice with the userapprentice with the user

QuestionsQuestions When does the first task in the process happen?When does the first task in the process happen? What triggers it?What triggers it? Who does it?Who does it? What information does the person have when the What information does the person have when the

task begins?task begins? What are the major steps in the task?What are the major steps in the task? What information comes out of it?What information comes out of it? Who is the next person in the chain of the process?Who is the next person in the chain of the process?


Example - Process map for a travel Example - Process map for a travel agent’s typical dayagent’s typical day

Interacting with the user – Interacting with the user – Condensed Ethnographic InterviewCondensed Ethnographic Interview

SynopsisSynopsis Use the results of Use the results of semi-structured interviewssemi-structured interviews to to

guide observationsguide observations AdvantagesAdvantages

This technique is This technique is considerably shorter than considerably shorter than some of the other techniquessome of the other techniques, because the , because the interviews scope what you observe;interviews scope what you observe;

It also It also limitslimits the data you are able to collectthe data you are able to collect Level of effortLevel of effort

It requires It requires a medium level of efforta medium level of effort to conduct to conduct interviews and then observe users and collect interviews and then observe users and collect artifactsartifacts

Interacting with the user – Interacting with the user – Condensed Ethnographic InterviewCondensed Ethnographic Interview

Employs the standardization and Employs the standardization and focus of a focus of a semi-structured interviewsemi-structured interview along with the context of along with the context of observations and artifactsobservations and artifacts

Characterized as Characterized as “top-down”,“top-down”, because the interviews form a because the interviews form a general framework from which to general framework from which to interpret specific observations.interpret specific observations.

Interacting with the user – Interacting with the user – Discount User Observation (DUO)Discount User Observation (DUO)

SynopsisSynopsis One facilitator interviews the user and takes notes while One facilitator interviews the user and takes notes while

another facilitator another facilitator takes pictures of everythingtakes pictures of everything AdvantagesAdvantages

You can You can quickly reconstructquickly reconstruct the session through the session through time stamped notes and digital picturestime stamped notes and digital pictures to verify to verify your observations with usersyour observations with users

Level of effortLevel of effort It It takes two investigatorstakes two investigators to focus on their specific to focus on their specific

tasks of either interviewing or taking pictures and notestasks of either interviewing or taking pictures and notes It will It will take additional time/effort to combine notes take additional time/effort to combine notes

with digital photos and put them in a multimedia with digital photos and put them in a multimedia presentation tools presentation tools to review with users.to review with users.

It takes It takes additional effort to validate observationsadditional effort to validate observations with user afterward.with user afterward.


Method supplementsMethod supplements Artifact walkthroughsArtifact walkthroughs Incident diariesIncident diaries Observing while you are not Observing while you are not

presentpresent

Method supplements – Method supplements – Artifact Artifact walkthroughswalkthroughs

SynopsisSynopsis Collect all the artifactsCollect all the artifacts used by participants used by participants

and determine and determine what triggers their usewhat triggers their use, , when they are usedwhen they are used, and for , and for whatwhat

AdvantagesAdvantages Quick and easyQuick and easy to conduct to conduct

Level of effortLevel of effort Low level of effortLow level of effort to review artifacts with to review artifacts with

participants and make/collect copies of participants and make/collect copies of themthem

Method supplements – Method supplements – Artifact Artifact walkthroughswalkthroughs

Quick and easyQuick and easy to conduct to conduct but but provide indispensable dataprovide indispensable data

““Artifacts” are Artifacts” are objects or itemsobjects or items that that users use to complete their tasks or users use to complete their tasks or that result from their tasksthat result from their tasks

You have to You have to sign the company’s sign the company’s confidential disclosure agreementconfidential disclosure agreement promising that you will promising that you will keep all data keep all data collected confidentialcollected confidential



presentpresent

Method supplements – Method supplements – Incident diariesIncident diaries

SynopsisSynopsis Worksheets the user takes home or to work Worksheets the user takes home or to work to collect to collect

ongoing dataongoing data AdvantagesAdvantages

No observation is requiredNo observation is required Able to Able to understand more issuesunderstand more issues than what can be than what can be

observed in the lab or during a single visitobserved in the lab or during a single visit Level of effortLevel of effort

Low effort levelLow effort level to create and distribute diariesto create and distribute diaries Moderate effortModerate effort to analyze data across multiple to analyze data across multiple

diariesdiaries. There will be a . There will be a time lagtime lag between when you between when you distribute the diaries and when you receive the datadistribute the diaries and when you receive the data

You are depending on the participantsYou are depending on the participants to follow- to follow-thorough.thorough.

Method supplements – Method supplements – Incident diariesIncident diaries

Incident diaries allow you to Incident diaries allow you to collectcollect ongoing data from usersongoing data from users

Participants are provided with Participants are provided with a notebook a notebook containing worksheetscontaining worksheets to be completed to be completed

They are given to users to They are given to users to keep track of keep track of issuesissues they encounter while using a they encounter while using a productproduct

The worksheets may ask usersThe worksheets may ask users Describe a problem or issue they encounteredDescribe a problem or issue they encountered How they solved itHow they solved it How troublesome it wasHow troublesome it was



presentpresent

Method supplements – Method supplements – Observing Observing while you are not presentwhile you are not present

SynopsisSynopsis Recording users in action when space, time, or Recording users in action when space, time, or

restrictions prevent you from being there in restrictions prevent you from being there in personperson

AdvantagesAdvantages If you have multiple video cameras, you can view If you have multiple video cameras, you can view

several users simultaneouslyseveral users simultaneously Level of effortLevel of effort

Low effortLow effort to to set up cameras and recordset up cameras and record Moderate effortModerate effort to to meet with user again and meet with user again and

review tapesreview tapes Moderate effortModerate effort to to categorize and index behaviorcategorize and index behavior

Method supplements – Method supplements – Observing Observing while you are not presentwhile you are not present

You can You can observe users even when observe users even when you are not presentyou are not present by setting up a by setting up a video camera and then leavingvideo camera and then leaving

Excellent way to Excellent way to understand detailed understand detailed stepssteps

Identify the type of study Identify the type of study to Conductto Conduct

decision diagramdecision diagram

Thank you~Thank you~

observing users presenter : jungmin shin & jaeyoung lee

Documents

observation slide

dirty observation

observation approaches

paradigms goals

field studies observers

ethnography slide

field studies insider

contents goals