the contextual wizard of oz
DESCRIPTION
The Contextual Wizard of Oz - Using the Wizard of Oz methodology in and with context for User Experience PrototypingTRANSCRIPT
Human Computer Interaction & Usability Unit http://icts.sbg.ac.at
Doris Zachhuber, Thomas Grill, Ondrej Polacek, Manfred Tscheligi !
mailto:[email protected] 12.11.2012
The Contextual Wizard of Oz
1
● - Wizard of Oz - Contextual - Framework / 23
The Wizard of Oz
!2
1
● - Wizard of Oz - Contextual - Framework / 23
The Wizard of Oz
!3
• Context: ! natural language interfaces ! from futuristic to UbiComp appliances
• Problems: ! Complexity of systems
Two wizard scenarios for complex systems • Goal for a wizard system:
! Bridging the gap between the wizard’s role and the system implementation
The Wizard of Oz method ... is a research experiment in which subjects interact with a (computer) system that subjects believe to be autonomous, but which is actually being operated or partially operated by an unseen (hidden) human being.
2
● - Wizard of Oz - Contextual - Framework / 23
Contextual ...
• Taking context into mind ! Study scenarios ! Prototyping ! Simulating context !
➡ ... a system that allows to simulate contextual parameters
!4
General statements: !
• it is done anyway ... • everybody does it ... • such systems already exist ... !
does such a system exist?
2
● - Wizard of Oz - Contextual - Framework / 23
Contextual ...
!5
Table 1: Survey and Analysis of Wizard of Oz tools
Funct-ionality
Wizardtool
Config Simu-lation
Data Domain
Name Applica
tion
spec
ific(*
)
Pro
toty
ping
Wizard
ing
Mobile
Sta
tionary
Gra
phical
Multiple
Config.
Wizard
–Auto
-Config
Pro
toty
pe/
System
Contextu
al
Logging
Rea
ltim
eControl
Scree
nCaptu
re
Analysis
Map
Support
Stu
dysu
pport
General applicable (G)HRI (H)Mathematics (M)Mobile (Mo)Multimodal (MM)LBS (L)PC application (P)Speech/Linguistik (S)Text Analysis (T)UbiComp (U)
DART A x x x x x x x x x x U,LiCAP G x ? x ? x x U,LOpenWizard D x x x x x x x x x U,MM,MoConWIZ G x x x x p x x x x x x p p x GSketchWizard A x x x x ? x x MoWebWOZ G x x x ? ? x MoCSLU Toolkit D x x x x x x SDiaWOz-II D ? x x x x x MHumaine Woz D x x x x x x T,SPolonius A x x x x x HSuede D x x x x x x x x SWizard of Oz 2 D x x x x x x SNEIMO A x x x x SMuMoWOz D x x x x x x U,MM,MoThe Mobile Oracle G x x GActiveStory S x x x x x x x PTopiary D x x x x x LWOEB G x x x x Mo,MM
Legend:x . . . available, p . . . planned feature, ? . . . could not be evaluated
* A . . . developed for one application, D . . . developed for one domain, G . . . generic application
in the study context (e.g. navigation route, simulated fac-tory, smart home, or sits in a car, etc.) The wizard ob-serves the user interacting with a mobile WOz Prototypeby means of the Mobile Wizard as well as through obser-vations and is able to react to the user’s actions as well asthe user’s actual context. Figure 2 depicts an example of aWOz study scenario where a human wizard is part of themobile study context and uses the Mobile Wizard to controla WOz prototype as well as contextual objects located inthe scenario. The Mobile Wizard connects to the CIF and isable to simulate functionality of the WOZ Prototype basedon the scenario the wizard observes and controls. The Mo-bile Wizard further allows the wizard to control contextualobjects that are objects located in the actual study context.Examples for this are e.g. tra�c lights in the parking area,door controls, locks, and gates. The mobile tool further al-lows the human wizard to observe the user’s position andlog data like GPS data, noise level, etc. automatically viathe CIF’s functionality. In addition the wizard is providedwith a note-taking functionality that allows to take notesabout important events.
ConWIZ – Mobile Wizard.The Mobile Wizard is a mobile application that is capableof controlling the simulation of a WOz prototype as wellas of contextual objects. It is implemented as an Androidapplication designed to run on devices of di↵erent sizes likesmartphones or tablets.Figure 3 depicts the basic interaction concept of the MobileWizard tool. After starting the tool it connects to the Con-WIZ server and the human wizard is provided with a listof configured WOz studies. After selecting the particularstudy the study configuration is loaded from the ConWIZ
Contextual Scenario
Factory building
C F
ConnectorConnector
Scenario - Corridor
Corridor Light
Navigation system
Scenario - Main Plaza
Traffic Light
Navigation System
Door Lock
CIF Wizard
Left Right
Back
Forward
Stop Go
Turn left Turn right
Turn right 90ºTurn left 90º
NotesNotes
human wizard
Parking
Main building
Park
Gate
Gate
Door
Door Main entrance
Door
Navigation
Destination
Volume
Emergency
Accident
WOz Prototype
observes
Contextual Scenario
Factory buildingParking
Main building
Park
Gate
Gate
Door
Door Main entrance
Door
Contectual Interaction Framework
ConWIZ Server
ConWIZ Server controls WOz prototype
Mobile Wizard connects to and
communicates with ConWIZ
server
Mobile Wizard
CIF communicates with contextual
objects
ConWIZ Server controls- contextual Objects via CIF
Figure 2: A ConWIZ – WOz scenario in a mobilecontext.
Server and provides the user with a list of configured scenar-ios. Each scenario contains a number of controllable objectsrepresenting specific parameters either of a WOz prototypeor a contextual object. Such parameters of a WOz proto-type can be for example the switching of states, displayingof specific results based on a user’s interaction, or the simu-lation of a more complex simulation based on a user’s inter-action with e.g. a mobile device or a static PC. RegardingUbiComp scenarios the simulation of WOz prototype pa-rameters are not limited to computers but also are extendedto prototypes based on programmable microprocessors con-trolling di↵erent actuators like an ambient light or a doorlock. This links to the simulation of contextual parame-ters where specific contextual situations can be simulatedby means of sensors and actuators. While sensors provide
[1] Thomas Grill, Ondrej Polacek, Manfred Tscheligi. Conwiz: A tool supporting contextual wizard of oz simulation. Proc. of MUM 2012. ACM, 2012.
Table 1: Survey and Analysis of Wizard of Oz tools
Funct-ionality
Wizardtool
Config Simu-lation
Data Domain
Name Applica
tion
spec
ific(*
)
Pro
toty
ping
Wizard
ing
Mobile
Sta
tionary
Gra
phical
Multiple
Config.
Wizard
–Auto
-Config
Pro
toty
pe/
System
Contextu
al
Logging
Rea
ltim
eControl
Scree
nCaptu
re
Analysis
Map
Support
Stu
dysu
pport
General applicable (G)HRI (H)Mathematics (M)Mobile (Mo)Multimodal (MM)LBS (L)PC application (P)Speech/Linguistik (S)Text Analysis (T)UbiComp (U)
DART A x x x x x x x x x x U,LiCAP G x ? x ? x x U,LOpenWizard D x x x x x x x x x U,MM,MoConWIZ G x x x x p x x x x x x p p x GSketchWizard A x x x x ? x x MoWebWOZ G x x x ? ? x MoCSLU Toolkit D x x x x x x SDiaWOz-II D ? x x x x x MHumaine Woz D x x x x x x T,SPolonius A x x x x x HSuede D x x x x x x x x SWizard of Oz 2 D x x x x x x SNEIMO A x x x x SMuMoWOz D x x x x x x U,MM,MoThe Mobile Oracle G x x GActiveStory S x x x x x x x PTopiary D x x x x x LWOEB G x x x x Mo,MM
Legend:x . . . available, p . . . planned feature, ? . . . could not be evaluated
* A . . . developed for one application, D . . . developed for one domain, G . . . generic application
in the study context (e.g. navigation route, simulated fac-tory, smart home, or sits in a car, etc.) The wizard ob-serves the user interacting with a mobile WOz Prototypeby means of the Mobile Wizard as well as through obser-vations and is able to react to the user’s actions as well asthe user’s actual context. Figure 2 depicts an example of aWOz study scenario where a human wizard is part of themobile study context and uses the Mobile Wizard to controla WOz prototype as well as contextual objects located inthe scenario. The Mobile Wizard connects to the CIF and isable to simulate functionality of the WOZ Prototype basedon the scenario the wizard observes and controls. The Mo-bile Wizard further allows the wizard to control contextualobjects that are objects located in the actual study context.Examples for this are e.g. tra�c lights in the parking area,door controls, locks, and gates. The mobile tool further al-lows the human wizard to observe the user’s position andlog data like GPS data, noise level, etc. automatically viathe CIF’s functionality. In addition the wizard is providedwith a note-taking functionality that allows to take notesabout important events.
ConWIZ – Mobile Wizard.The Mobile Wizard is a mobile application that is capableof controlling the simulation of a WOz prototype as wellas of contextual objects. It is implemented as an Androidapplication designed to run on devices of di↵erent sizes likesmartphones or tablets.Figure 3 depicts the basic interaction concept of the MobileWizard tool. After starting the tool it connects to the Con-WIZ server and the human wizard is provided with a listof configured WOz studies. After selecting the particularstudy the study configuration is loaded from the ConWIZ
Contextual Scenario
Factory building
C F
ConnectorConnector
Scenario - Corridor
Corridor Light
Navigation system
Scenario - Main Plaza
Traffic Light
Navigation System
Door Lock
CIF Wizard
Left Right
Back
Forward
Stop Go
Turn left Turn right
Turn right 90ºTurn left 90º
NotesNotes
human wizard
Parking
Main building
Park
Gate
Gate
Door
Door Main entrance
Door
Navigation
Destination
Volume
Emergency
Accident
WOz Prototype
observes
Contextual Scenario
Factory buildingParking
Main building
Park
Gate
Gate
Door
Door Main entrance
Door
Contectual Interaction Framework
ConWIZ Server
ConWIZ Server controls WOz prototype
Mobile Wizard connects to and
communicates with ConWIZ
server
Mobile Wizard
CIF communicates with contextual
objects
ConWIZ Server controls- contextual Objects via CIF
Figure 2: A ConWIZ – WOz scenario in a mobilecontext.
Server and provides the user with a list of configured scenar-ios. Each scenario contains a number of controllable objectsrepresenting specific parameters either of a WOz prototypeor a contextual object. Such parameters of a WOz proto-type can be for example the switching of states, displayingof specific results based on a user’s interaction, or the simu-lation of a more complex simulation based on a user’s inter-action with e.g. a mobile device or a static PC. RegardingUbiComp scenarios the simulation of WOz prototype pa-rameters are not limited to computers but also are extendedto prototypes based on programmable microprocessors con-trolling di↵erent actuators like an ambient light or a doorlock. This links to the simulation of contextual parame-ters where specific contextual situations can be simulatedby means of sensors and actuators. While sensors provide
2
● - Wizard of Oz - Contextual - Framework / 23
Contextual Wizard of Oz
• Goal: ! Usable and configurable system that supports the engineer through
prototyping and evaluation !
! Generic interface. No reimplementation of the wizard is needed. !
! Reduce the workload of the human wizard’s Support of multiwizard scenarios !
! Applicable in multiple contexts • Desktop • Mobile • UbiComp • Field and Lab
!6
2
● - Wizard of Oz - Contextual - Framework / 23
Contextual Wizard of Oz
!7
Contextual Scenario
Factory buildingParking
Main building
Park
Gate
Gate
Door
Door Main entrance
Door
WOz PrototypeC F
ConnectorConnector
Contectual Interaction Framework
ConWIZ Server
ConWIZ Server controls- contextual Objects via CIF
Scenario - Corridor
Corridor Light
Navigation system
Scenario - Main Plaza
Traffic Light
Navigation System
Door Lock
CIF Wizard
Left Right
Back
Forward
Stop Go
Turn left Turn right
Turn right 90ºTurn left 90º
NotesNotes
human wizard
observes
Contextual Scenario
Factory buildingParking
Main building
Park
Gate
Gate
Door
Door Main entrance
Door
Mobile Wizard connects to and
communicates with ConWIZ
server
Mobile Wizard
ConWIZ Server controls WOz prototype!
CIF communicates with contextual objects
3
● - Wizard of Oz - Contextual - Framework / 23
Contextual Wizard of Oz
Interaction Framework
Wizard FrameworkSim. Control
ModuleReal World
Control Module
- start, stop simulation- set scenes- set parameters of objects
- control the behaviour of real world objects through communication with the interaction platform
integrates & controls
controls
Wizard of Oz Communication
Protocol
Visualization Framework
!8
3
● - Wizard of Oz - Contextual - Framework / 23
Motivation & Goals
•Conduct user studies in the (simulated) target context ! Analyze the interplay of user interaction and selected context
parameters ! Change selected context parameters and adapt interactive systems
(following the ubiquitous computing approach) to evaluate changes in user behavior !
•Rapid prototyping ! Speed up the development and configuration of interactive systems
and the simulated context in which they are applied. ! Reusing existing software components
!9
3
● - Wizard of Oz - Contextual - Framework / 23
Contextual Interaction Framework
Actuator X
Actuator X
Interaction Framework
Sensor 1
Sensor 2
Sensor X
Visualization Framework
defines scenario
Visualisation Audio Sim. Ext. Interf. Sim.
WOz protocol
Interaction/State Analyzer
PlugIns
Control Module
Feedback Generator
PlugIns
Screen X ...
Tracking System X input
output - feedbackAudio X
Control Module
represented through output devicesi.e. screens, speaker and other actuators
Logging Platform
Scenario Implementation
observation/evaluation
Object libraryAgent libraryScene library
Wizard FrameworkSim. Control
ModuleReal World
Control Module
logging may enhance usability evaluation and analysis of the particular data
- start, stop simulation- set scenes- set parameters of objects
- control the behaviour of real world objects through communication with the interaction platform
!10
3
● - Wizard of Oz - Contextual - Framework / 23
Mobile Wizard
Contextual Interaction Framework
WOz Protocol
Context Simulation WebServer
FileServerDB Server
Ext. Context Simulation
(3D App., etc.)
Mobile Wizard
ConWIZ Server
Study Configuration
WOz Protocol
Network Communication
Wizard UI
TCP
Scenario Configuration / Study Editor
CIF CoreUtilities
TCP
TCP
Contextual Interaction Framework
!11
3
● - Wizard of Oz - Contextual - Framework / 23
Contextual Interaction Framework
• CIF CORE ! SOA ! PlugIns, Wiring, Datatypes, ... ! PlugIn Configuration support !
• CIF Study Editor ! Wiring based graphical editor ! Prototyping ! Save and load configurations ! Access to CIF Library (currently in development) ! Study Support !
• CIF Wizard Server (ConWIZ) ! CIF Wizard protocol
!12
3
● - Wizard of Oz - Contextual - Framework / 23
CIF Study Editor
• Wiring based graphical editor
• Prototyping • Save and load configurations
• Access to CIF Library (currently in development)
• Study Support
!13
3
● - Wizard of Oz - Contextual - Framework / 23
CIF Study Editor- + IFW Study Editor: OG Study Setup
Producers / SensorsRFID ReaderDistancesensorMicrophone
Bundle Library
Consumers / ActuatorsConverters
DB AnalyzerDatatype Converter
XYZ...
System bundles
Bundle name Status
All Active
started
started
Library Inspector Align Diagram Zoom in Zoom out Zoom fit
Distance Sensor
float float
Activity Meter
floatstringfloatint
DB Logger ⚙
Camera
raw image raw image
Head Motion Analyser
stringstringfloatint
DB Logger ⚙
wzTypewzTypewzType
Wizard
wzType
⚙
wzType
Wizard Data Converter
stringbool
int
bool
Speaker
Light Sensor
float wzType
Wizard Data Converterstringboolfloat
!14
3
● - Wizard of Oz - Contextual - Framework / 23
ConWIZ Mobile Wizard
• Configurable interface • No adoption of the application needed for different study concepts
• Dynamic communication protocol • Mobile solution
!15
3
● - Wizard of Oz - Contextual - Framework / 23
Wizard of Oz Mobile App
•Wizard of Oz – Mobile Client ! Control and triggering of contextual events on a mobile device ! Implemented as an app on android tablet ! Communicates with the CIF Wizard Server
GUI Draft
start
end
connect towizard server
dis-connected
study selection
scene selection
selectstudy
wizardscreen
selectscene
backbackbackexit
!16
3
● - Wizard of Oz - Contextual - Framework / 23
Wizard of Oz Mobile App
•Check and modify contextual parametersAvailable Wizard
ObjectsState of the
object
Modify state
!17
3
● - Wizard of Oz - Contextual - Framework / 23
Wizard of Oz Mobile App•Invoke Prototype functionalities
!18
3
● - Wizard of Oz - Contextual - Framework / 23
Scenario: factory labair lock
MS 1
MS 2
HS 1
MS 3
Wizard
measurement station
impl
anat
ion
TrolleyWafer Box
Wafer Box
Wafer Box
table
RFID Reader
RFID Reader
RFID Reader
RFID Reader
RFID Tags
RFID Tags
RFID Tags
Flatter
storage rack
delivery rack
Light KeyboardKeyboard
KeyboardKeyboard
Place for
Trolley
Operator Guideev
alua
tor a
nd w
izard
spa
ce
!19
3
● - Wizard of Oz - Contextual - Framework / 23
Scenario: factory lab
• CIF in the factory lab • Human Wizard uses ConWIZ
! Control prototype (machines)
! Control simulation of contextual parameters (i.e. feedback conditions, vibrations, etc.) !
• Evaluation of ! workflow in a factory
context ! influence of ambient
alerting modes
!20
[1] Ewald Strasser, Astrid Weiss, Thomas Grill, Sebastian Osswald, Manfred Tscheligi. Combining implicit and explicit methods for the evaluation of an ambient persuasive factory display. In Proc. of AMI 2012, 2012.
3
● - Wizard of Oz - Contextual - Framework / 23
Scenario: Mobile
• CIF running on a Laptop • Human Wizard uses ConWIZ
! Control prototype ! Control simulation of
environmental parameters !
• Evaluation of ! navigation commands ! usability of mobile wizard
!21
[1] Thomas Grill, Ondrej Polacek, Manfred Tscheligi. Conwiz: A tool supporting contextual wizard of oz simulation. Proc. of MUM 2012. ACM, 2012.
ConWiz - Mobile Wizard Evaluation
3
● - Wizard of Oz - Contextual - Framework / 23
Contextual Interaction Framework
• Java based multi-platform enabled framework • OSGi Equinox !
• Available for download as !
! Developer Library ! Standalone version
• Starts with CIF Study editor ! Mobile Android App
!22
3
● - Wizard of Oz - Contextual - Framework / 23
• General Information ! Download, Documentation,
Support, ... !
• Community Base ! Forum !
• CIF PlugIn Library ! Place for exchanging
developed functionalities
CIF Web
!23
http://cif.icts.sbg.ac.at/
● Questions?
grazie mille!
* Contact
Human Computer Interaction & Usability Unit [email protected]
HCI & Usability Unit ICT&S Center, University of Salzburg Sigmund-Haffner-Gasse 18 5020 Salzburg, Austria [email protected] !!!!Dr. Thomas Grill [email protected]
●
● - Topic 1 - Topic 2 - Topic 3 - Topic 4 - Topic 5 / 23!26
This is the end ...
●
● - Topic 1 - Topic 2 - Topic 3 - Topic 4 - Topic 5 / 23
Future Work
• More customizable wizard interface • Less navigation-based interaction • More scenario-based interaction
!27