h"p://hci.uni-­‐konstanz.de  

Prof. Dr. Harald Reiterer University of Konstanz

Chair for Human-Computer Interaction Harald.Reiterer@uni-konstanz.de

hci.uni-konstanz.de

University of Konstanz

Blended Libraries  

Future  user  interfaces  concepts  to  support  informa0on  seeking  and  

collabora0on  

 2  

Introduc7on  •  University  of  Konstanz,  Department  

of  Computer  &  Informa0on  Science,  Head  of  the  Research  Group  Human-­‐Computer  Interac0on  (since  1997)    

•  Research  Topics:  Interac0on  Design,  Usability  Engineering,  Informa0on  Visualiza0on  

•  Main  Goal:  Improving  the  naturalness  of  the  interac7on  between  human  and  computer  

– Mul0modal  Interac0on  –  Reality-­‐based  Interac0on  

 3  

UIs  for  Informa7on  Seeking  Systems  (ISS)  

•  INSYDER  (ISS  for  the  Web)  –  EU  Project  Eureka  No.  29232  (1999-­‐2001)  

•  INVISIP  (ISS  for  Geo-­‐Data)  –  EU  Project  No.  IST-­‐2000-­‐29640  (2002-­‐2004)  

•  Mobile  Informa7on  Visualiza7on  (ISS  for  mobile  devices)  –  DFG  PhD  Graduate  Program  (2005-­‐2007)  

•  MedioVis  (ISS  for  digital/mul0media  libraries)  –  DFG  LIS  4-­‐54281(3)  (2007-­‐2009)  

•  Blended  Library  -­‐  Ministerium  für  Wissenschaa,  Forschung  und  Kunst,  Baden-­‐Würcemberg  (2011-­‐2013)  

 

more  informa0on:  hcp://hci.uni-­‐konstanz.de  

 4  

Mo7va7on  Poor  Usability  of  OPACs  (2007)  

 5  

MedioVis  Demo  of  System  in  Use  

Integration of Multimedia Metadata and Web Services

 6  

Library of the Future –  Blend  Physical  &  Digital  Library  

digital  library   physical  library  

status-­‐quo    co-­‐existance  

vision    co-­‐opera0on  

 7  

Blended  Library  –  Domains  of  Design  Individual  Interac7on:  Blending  real  world  interac0on  &  objects  (e.g.  scribbling  text,  sketching  objects,  grasping  tokens)  with  digital  ac0ons  &  representa0ons  (e.g.  to  move  digital  books  or  movies,  touch  items  of  a  facet  token  value  wheel).    Social  Interac7on  &  Communica7on:  Different  user  can  interact  at  the  same  0me  on  an  equal  manner  considering  real-­‐word  social  conven0ons  of  communica0on  (e.g.  mul0-­‐touch  displays  allows  mul0user  interac0on;  tokens  could  be  grasped  equally).    Workflow:  Fluid  change  between  real-­‐world  workflows  and  computer-­‐supported  workflows  (e.g.  search  in  a  bookshelf  –  search  on  a  Tabletop).      Physical  Environment:  Blending  the  power  of  real-­‐world  devices,  furniture,  rooms  &  buildings  with  the  power  of  virtual  means  (e.g.  combining  tables,  chairs,  walls,  floor,  ceiling,  light,  sound  with  mul0-­‐touch  planar,  spherical,  cylindrical,  curved  displays);  new  arrangements  of  place  and  space.    

 8  

What  kind  of  UI  for  different  Displays?  Large  –  small  

Different  Input  modali0es    (e.g.  Touch,  Stylus,    Keyboard,  Mouse)  

Real-­‐0me  Synchroniza0on  

?  

 9  

Zoomable  Object-­‐Oriented  Informa7on  Landscape  

 10  

Query  by  Real  World  Objects,  e.g.  DVD  

 11  

Query  by  Real  World  Objects,  e.g.  DVD  

 12  

Blended  Library  –  Coopera7ve  Search  with  Search  Tokens  

 13  

Blended  Library  –  Coopera7ve  Search  with  Search  Tokens  

 14  

User  Study    -­‐  Se_ng  

Par7cipants:  75  par0cipants,  tested  in  25  randomly  selected  3-­‐person-­‐groups  (triads)  Between-­‐Subject  Design  (12  +  13  groups)  Two  UI  Condi7ons:  -­‐  Synchronized  PC  Interface  -­‐  Search  token  UI:  Physical  search  token  +      Mul0-­‐touch  Tabletop  

Research  Ques7on:    What  impact  has  a  search  token  UI  on  the  •  Interac0on  Strategy,  •  Communica0on,  •  Roles  of  Collabora0on?  

 

 15  

Selected  Results  

•  Interac7on:  Tokens  featured  more  parallel  interac0on.  •  Interac7on:  Tokens  enhance  the  coopera0on  between  the  par0cipants,  they  

even  share  their  tokens!  •  Verbal  Communica7on:  no  significant  differences.  •  Non-­‐Verbal  Communica7on:  With  the  Search  Token  UI  gestures  and  posture  

were  perceived  without  needing  to  look  up  from  the  display.  •  Roles  of  Collabora7on:  5  different  roles  iden0fied.  Search  Token  UI  allowed  

ac0ve  lead  users  to  take  on  a  more  dominant  role  within  the  groups.  Even  the  passive  group  members  showed  a  more  ac0ve  role  compared  with  the  passive  users  in  the  synchronized  PC  condi0on.  

 16  

Tabletops  for  Co-­‐Located  Collabora7ve  Faceted  Search  

 17  

Tabletops  &  Collabora7ve  Search:  Why?  •  poten0ally  great  support  of  mechanics  of  collabora7on  

[Gutwin  &  Greenberg  2000]  

•  closer  face-­‐to-­‐face  collabora7on  and  more  equitable  working  style    [Rogers  &  Lindlay  2004]  

•  increased  awareness  and    be"er  group  work  experience    [Amershi  &  Morris  2008]  

 18  

Facet-­‐Streams  Approach  •  Hybrid  Surfaces  [Kirk  et  al.  2009]  

–  Combining  mul0-­‐touch  interac0on  with  tangible  interface  elements  

 

•  Faceted  Search  [Hearst  2009]  –  itera0vely  filtering  the  whole  informa0on  space  based  on  metadata,  instead  of  popula0ng  a  result  set  based  on  keywords  

 19  

Facet-­‐Streams  Approach  

Visual  &  tangible  filter/flow  metaphor  for  Boolean  logic    Redesign  of  [Young  &  Shneiderman  1993,  Hansaki  et  al.  2006]  for  tabletops  

 20  

Research  Ques7ons  •  Does  Facet-­‐Streams  turn  collabora0ve  product  search  into  a  

fun  and  social  experience  with  increased  group  awareness?  

•  Can  Facet-­‐Streams  support  the  great  variety  of  different  search  strategies  and  collabora0on  styles  in  different  teams  with  a  simple  but  flexible  design?    

•  Can  Facet-­‐Streams  harness  the  expressive  power  of  facets  and  Boolean  logic  without  exposing  users  to  complex  formal  nota0ons?    

 21  

Study  1  -­‐  Facet  Streams  in  collabora7ve  use  •  72  par0cipants  in  24  groups  of  three  •  Students  and  faculty  staff  from  a  variety  of  fields  (only  2  

computer  science  students)  

 22  

The  Web-­‐Interface  

 23  

Task  •  Goal:  Facilitate  compromise  nego,a,on  in  a  controlled  

environment  

 24  

Results  -­‐  Strategies  

 25  

Results  –  Study  1  • Facet-Streams equally effective as established Web

designs for faceted navigation (although it introduces novel and unfamiliar hybrid interaction techniques and visual metaphors).

• Users perceived using Facet-Streams as a fun experience and considered its design as innovative.

•  Increased awareness and better mutual support among collaborators was observed.

• Variety of different search strategies and collaboration styles could be realized with our design.

• Seamless transitions between tightly-coupled collaboration and loosely-coupled parallel work.

 26  

Study  2  –  Comprehensiveness  of  Boolean  logic  

•  7  par0cipants  (no  computer  science  students)  •  Individual  sessions  (no  group  interac0on)  •  Focus:    

–  Task  1:  Can  people  understand  and  learn  our  visual  representa0on  of  Boolean  logic  (without  being  made  aware  that  this  is  Boolean  logic!)?    

–  Task  2:  Are  they  able  to  formulate  a    complex  and  evolving  query  based  on  natural  language  instruc7ons?  

 27  

Study  2  -­‐  Task  1    3  networks  3  Hotels  per  network  

4  spots  per  hotel  

=36  Ques0ons  

Can  people  understand  and  learn  our  visual  representa0on  of  Boolean  logic  (without  being  made  aware  that  this  is  Boolean  logic!)?    

 28  

Results    Study  2  –  Task  1  

Correct   Wrong  

Total   234  (92,9%)   18  (7,1%)  

AND   101  (96,2%)   4  (3,8%)  

OR   133  (90,5%)   14  (9,5%)  

 29  

Study  2  –  Task  2  •  Are  people  able  to  formulate  a    complex  and  evolving  query  based  on  

natural  language  instruc7ons?  

•  Experimenter:  Role  of  a  costumer  –  User:  Role  of  a  travel  agent  

–  „I  would  be  interested  in  a  hotel  with  a  price  between  50  and  120  EUR,  a  very  high  room  quality,  and  at  least  4  stars.“  

–  „It  should  be  in  Spain.“  –  „Lets  include  those  hotels,  that  have  a  high  loca0on  quality  instead  of  

a  high  room  quality  as  an  alterna0ve.  They  should  s0ll  have  the  other  features.“  

–  …  –  Requirements:  Zero  Hit,  AND,  OR,  temporary  deac0va0on,  mul0ple  

output  streams,  parallel  streams  

 30  

Results  Study  2  -­‐  Task  2  •  6  of  7  correctly  integrated  the  OR  connec0on  

 

   

•  Boolean:  (room  quality  =  4-­‐6  OR  loca,on  quality  =  5-­‐6)  AND  (hotel  stars  =  4-­‐5)  AND  (country  =  Germany  OR  country  =  Spain)  AND  (features  =  Bar+Pool)  

 31  

Results  –  Study  2  • Users  were  able  to  quickly  learn  and  apply  our  visual  metaphor  for  Boolean  logic  

• Users  succeeded  in  formula0ng  complex  Boolean  queries  based  on  natural  language  instruc0ons  

 32  

Natural  User  Interfaces  “The goal is to make computer interaction more like interacting with the real, non-digital world.” Reality-­‐Based  Interac0on  (Jacob  et  al.  2008)  

 

 

 

 

 

 

User flicks screen to left or right to see next Album – all objects in the real word have spatial relationships.

Fast flick will keep contacts scrolling after finger has been removed, as if list has mass.

Natural gestures control the applications.

User do not share one single input device – Tabletop allows to draw more directly on existing social interaction skills.

 33  

Conclusion  •  Conclusions  •  Blended Library draw strength by building on users’ pre-existing

knowledge of the everyday, non-digital library world. •  It employ experiences of reality and makes the interaction with the

computer much more natural. •  It blends the benefits of digital libraries with the benefits of physical

libraries to make visiting and using a library a new experience.

[Microsoft Research UK 2008, Being Human: Human-Computer Interaction in the Year 2020]

 34  

Papers  and  Videos  available  on    our  Website  hci.uni-­‐konstanz.de    

   

Ques7ons?