understanding contextual factors in location-aware multimedia messaging

34
Understanding Contextual Factors in Location-aware Multimedia Messaging Abdallah “Abdo” El Ali Frank Nack Lynda Hardman

Upload: abdo-el-ali

Post on 05-Dec-2014

1.020 views

Category:

Technology


1 download

DESCRIPTION

A talk I gave at ICMI 2010, held in Beijing, China. The full paper reference is: El Ali, A., Nack, F. & Hardman, L. (2010). Understanding contextual factors in location-aware multimedia messaging. In Proceedings of the 12th international conference on Multimodal Interfaces, 2010, Beijing, China.

TRANSCRIPT

Page 1: Understanding Contextual Factors in Location-aware Multimedia Messaging

Understanding Contextual Factors in Location-aware Multimedia Messaging!

Abdallah  “Abdo”  El  Ali  

Frank  Nack  

Lynda  Hardman  

Page 2: Understanding Contextual Factors in Location-aware Multimedia Messaging

Outline!

I.  Introduc*on  

II.  Prototype  

III.  Diary  Study  

IV.  Results  

V.  Conclusions  &  Future  Work  

2 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 3: Understanding Contextual Factors in Location-aware Multimedia Messaging

  Ubiquitous  Compu*ng  promises  to  populate  our  daily  lives  with  specialized  ‘context-­‐aware’  services  that  enhance  our  experience  

    easier  

    friendlier  

    efficient  

  Loca*on-­‐awareness  (GPS)    Context-­‐awareness  (human  intent?)  

   

3

Motivation!

(Weiser, 1995)"

(Dey, 2002)"

Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 4: Understanding Contextual Factors in Location-aware Multimedia Messaging

! Case Study: Location-aware Multimedia Messaging (LMM)!

       Geo-­‐tagged  

mul*media:  photos,  text,  video,  audio    

     Made  at  a  loca*on  and  viewed  at  that  loca*on    

      Assump:on:  LMMs  can  

reflect  cultural  aspects  of  people’s  experiences  and  make  them  visible  at  loca*ons  

       Window  into  experiences?      

4

e.g.,  Photos  of  sunset,  tagged  with  sunset,  taken  at  *me  t  indicates  X  was  appreciating something

Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 5: Understanding Contextual Factors in Location-aware Multimedia Messaging

What is an Experience?!

  Memory  View:  Result  of  an  experience  process.  Experience  memory  consists  of  one  or  more  actors,  spa*otemporal,  social,  cogni*ve,  and  affec*ve  aspects  

   “Looking  at  these  photos  reminds  me  of  the  good  *mes  we  had  at  the  last  ICMI  conference  in  Beijing”  

   Post-­‐hoc  representa*on          Used  as  a  framework  to  understand  the  contextual  

factors  in  LMM      Emphasis  on  message  produc*on    

5

based on idea of episodic memory "(Tulving, 1993)

Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 6: Understanding Contextual Factors in Location-aware Multimedia Messaging

Questions!

1.  What  contextual  factors  are  involved  in  using  LMM  systems?  

2.  Can  these  factors  inform  the  study  and  design  of  future  LMM  systems?    

      Exploratory  study!  6 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 7: Understanding Contextual Factors in Location-aware Multimedia Messaging

Related Work!     GeoMedia  (Papliatseyeu  &  Mayora,  2008):         -­‐  permits  aaaching  mul*media  messages  (as  images,  audio  

  or    video)  to  loca*ons       -­‐  no  user  study  

  GeoNotes  (Persson  &  Fagerberg,  2002)  and  E-­‐graffi*  (Burrell  &  Gay,  2002):    

    -­‐  loca*on-­‐aware  systems  that  allow  users  to  leave  textual        messages  such  as  reminders  or  post-­‐it  notes  at  loca*ons  

    -­‐  extensively  studied  in  real-­‐world  usage  contexts       -­‐  focus  on  user  reac*ons  to  designed  systems       -­‐  tools  used  as  loca*on-­‐based  e-­‐mail  

  We  were  interested  in:       -­‐  whether  mul*media  messages  can  help  understand  experiences  

7 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 8: Understanding Contextual Factors in Location-aware Multimedia Messaging

Prototype!

      LMM  prototype  on  the  Android  mobile  device      

  Allows  annota*on  of  loca*ons  with  mul*media  messages  (drawings,  text,  photographs)  

Tool  to  acquaint  study  par*cipants  with  LMM  concept  

8 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 9: Understanding Contextual Factors in Location-aware Multimedia Messaging

Prototype: Message Creation!

Create     -­‐  Touch-­‐based  

drawing  (doodles)     -­‐  Wri*ng  text  

Snap       -­‐  Photographs  

9

Create at t1"

Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Engage at t0"

Page 10: Understanding Contextual Factors in Location-aware Multimedia Messaging

Prototype: Message Viewing!

  Explore       When  user  is  at  the  right  

posi*on  and  orienta*on,  s/he  can  view  the  message  

  Message  appears  as  an  Augmented  Reality  overlay  on  the  camera  view  

10

View at t2"

Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 11: Understanding Contextual Factors in Location-aware Multimedia Messaging

Diary Study !

   

  Longitudinal  (~1  week)  mul*modal  diary  method  

   

11

(Amin, 2009)"

Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 12: Understanding Contextual Factors in Location-aware Multimedia Messaging

Categorization Task!

   Needed  for  inter-­‐coder  reliability  (n=6)        

  Messages  categorized  according  to  both:  

    Domain:  What  is  the  message  about?                                          (e.g.,  entertainment,  architecture)  

   

      Task:  What  was  the  purpose  of  the  message?           (e.g.,  apprecia*on,  cri*cism)  

12 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 13: Understanding Contextual Factors in Location-aware Multimedia Messaging

Diary Study: Participants !

       

8  subjects  (6  m,  2  f)  aged  between  13-­‐27  (M=  23;  SD=  4.4)  

13 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 14: Understanding Contextual Factors in Location-aware Multimedia Messaging

Diary Study: Materials !

   

   8  custom-­‐designed  paper  diaries,  where  each  had  a  template  asking:  

  1.  Ques*ons  about  the  message  

  2.  Ques*ons  about  the  subject  and  her  context  

 Post-­‐study  interview  ques*ons  

14 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 15: Understanding Contextual Factors in Location-aware Multimedia Messaging

Diary Study: Materials(1) !

1.  Ques*ons  about  the  expression:          

    -­‐  Date         -­‐  Time       -­‐  Message  format  (drawing,  text,  photo,    -­‐  

  video,  audio  recording,  other)       -­‐  Title  of  message  

    -­‐  Public  or  private          

15 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 16: Understanding Contextual Factors in Location-aware Multimedia Messaging

Diary Study: Materials(2) !

2.  Ques*ons  about  the  subject  and  her  context:  

  Spa*otemporal  (Q1,  Q4)     Social  (Q5)     Affec*ve  (Q3)     Cogni*ve  aspects  (Q2,  

Q7,  Q8,  Q9,  Q10,  Q11)  

16 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 17: Understanding Contextual Factors in Location-aware Multimedia Messaging

Diary Study: Materials(3) !

      Interview  Ques*ons:    

          -­‐  Difficulty  filling  in  the  diary       -­‐  Media  preference              -­‐  Awareness  and  experience  of  past  week              -­‐  Desire  to  view  and  write  message  metadata  

           -­‐  Willingness  to  use  a  future  mobile  applica*on      

17 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 18: Understanding Contextual Factors in Location-aware Multimedia Messaging

Procedure!

                 Info  brochure      

                  LMM  Prototype  Demo    

          Diary  (2  mul*media  messages  per  day  for  1  week)    

        Post-­‐study  Interview    

18 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 19: Understanding Contextual Factors in Location-aware Multimedia Messaging

Results!

  Total  message  count:  110  

  Vo*ng  “winner-­‐takes-­‐all”  procedure  used  to  analyze  Categoriza*on  Task  results  (n=6)    

  If  equal  number  of  responses  to  two  dis*nct  categories  then  expression  classified  under  both    

      Categoriza*on  Task  results  directly  assimilated  into  diary  

study  results    

   

19 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 20: Understanding Contextual Factors in Location-aware Multimedia Messaging

Media Choice!

  “In  the  beginning,  it  was  photos,  and  during  the  week,  because  it  wasn't  that  interes*ng,  I  used  more  text"    

   

   

Text  (symbolic)  can  be  used  to  express  something  beyond  quali*es  of  the  loca*on  itself  

Songs  act  as  surrogates  for  the  memory  of  a  place  

20

Page 21: Understanding Contextual Factors in Location-aware Multimedia Messaging

Total = 114%"

21

Domain Ratings (N=6) for 110 Messages!

Page 22: Understanding Contextual Factors in Location-aware Multimedia Messaging

Total = 113%"

22

Task Ratings (N=6) for 110 Messages!

Page 23: Understanding Contextual Factors in Location-aware Multimedia Messaging

Spatiotemporal Aspects!

  Controlling  for  Public  Place,  many  messages  were  about  Ac*vity  Repor*ng  (39%)    

   microblogging  behavior  (e.g.,  Twiaer  feeds)    

  Most  Urban  expressions  fell  into  Aesthe*cs  (63%)  and  Apprecia*on  (49%)    

   *ght  correspondence  between  being  outdoors  and  aesthe*c  apprecia*on  

   

23

Page 24: Understanding Contextual Factors in Location-aware Multimedia Messaging

  Difference  between  public  and  private  messages  and  messages  made  alone  or  in  the  company  of  others    

   Messages  made  alone  were  also  made  public  (76%)        

24

Social Aspects!

Page 25: Understanding Contextual Factors in Location-aware Multimedia Messaging

Affective Aspects!

  Tendency  between  being  alone  and  nega*vely  valenced  mood  (60%)  

   Cathar*c  outlet  typical  of  web  2.0  social  behavior?    

25

(Russell, 1980)

Page 26: Understanding Contextual Factors in Location-aware Multimedia Messaging

Cognitive Aspects!

  Causal  rela*on  between  prior  ac*vity  &  message  crea*on:  35%  

  Interview:  Awareness  of  daily  environment?    

    All  reported  planning     behavior,  but  not  if  the  tool     is  embedded  in  daily  life  

26 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 27: Understanding Contextual Factors in Location-aware Multimedia Messaging

Viewing Metadata(1)!           Context  metadata  (mood,  companions,  event)  desired:  6  

subjects     Standard  metadata  (name,  date,  *me)  desired:  2  subjects      

  "Not  at  first  sight,  that  would  ruin  my  personal  view  of  their  message.  But  it  should  be  available  if  wanted...why  the  message  was  made,  what  did  the  person  want  to  express."  

   

27 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 28: Understanding Contextual Factors in Location-aware Multimedia Messaging

Viewing Metadata(2)!

  No*fica*on:       Filter  by  context:  7  subjects       Query:  1  subject         “If  I'm  walking,  then  I'd  like  to  search  myself,  but  if  I'm      

biking,  I'd  like  noEficaEon  of  what  there  is”      

   No*fica*on  should  depend  on  the  situa*on  subjects  are  in  to  avoid  interrup*on  

   Personaliza*on  best  considered  itself  context-­‐dependent?  

28 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 29: Understanding Contextual Factors in Location-aware Multimedia Messaging

Limitations!

      2  expressions  per  day  for  one  week  is  unnatural     Cogni*ve  effort    

  Availability  of  media  capture  devices     Instruc*ons  insufficient  

29 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions"

Page 30: Understanding Contextual Factors in Location-aware Multimedia Messaging

Implications!

  Predominant  domain  (Aesthe*cs,  Entertainment)  and  task  (Apprecia*on,  Ac*vity-­‐repor*ng)  categories  in  experience    capture  behavior  

  Applica*on  personaliza*on  (‘when’)  should  depend  on  and  adapt  to  the  user's  context  (‘what’)  

  Capturing  experiences  (memory  view)  vs.  the  experience  of  capture  (process/interac*on  view)  

     

30 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions!

Page 31: Understanding Contextual Factors in Location-aware Multimedia Messaging

Conclusions!

  Paper  diary  a  useful  low-­‐fidelity  mechanism  for  understanding  contextual  factors  in  LMM  

         Experience  memory  framework  not  very  useful  and  effortul  

to  analyze    clear  behavioral  paaerns  emerge  with  larger  datasets?    

31 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions!

Page 32: Understanding Contextual Factors in Location-aware Multimedia Messaging

Current / Future Work!

  Path  planning  &  POI  recommenda*on  service  based  on  social  media  content  (e.g.,  FlickR  photos)  

   Experience  as  Memory  View  

  Mul*modal  &  Crossmodal  feedback  in  place  and  path  recommenders  to  enhance  city  explora*on  

   Experience  as  Interac*on  View  

32 Introduction" " Prototype" " " Diary Study" " Results " " Conclusions!

Page 33: Understanding Contextual Factors in Location-aware Multimedia Messaging

Thanks  for  listening.  Ques:ons?  

(VISIT  ME  AT  MY  DOCTORAL  SPOTLIGHT  POSTER)

33 Website:  http://staff.science.uva.nl/~elali/

Page 34: Understanding Contextual Factors in Location-aware Multimedia Messaging

References!A.  Papliatseyeu  and  O.  Mayora  Ibarra.  Nailing  the  reality  with  GeoMedia:  loca*on-­‐

aware  mul*media  tags.  In  Proceedings  of  MobiMedia’08  Conference,  Oulu,  Finland,  July  2008.  ACM.  

A.  Amin,  S.  Townsend,  J.  Ossenbruggen,  and  L.  Hardman.  Fancy  a  drink  in  canary  wharf?:  A  user  study  on  loca*on-­‐based  mobile  search.  In  INTERACT  ’09:  Proceedings  of  the  12th  IFIP  TC  13  Interna*onal  Conference  on  Human-­‐Computer  Interac*on,  pages  736–749.  Springer-­‐Verlag,  2009.  

E.  Tulving.  What  is  episodic  memory?  Current  Direc*ons  in  Psychological  Science,  pages  67-­‐70,  1993.  

J.  Burrell  and  G.  K.  Gay.  E-­‐graffi*:  Evalua*ng  real-­‐world  use  of  a  context-­‐aware  system.  Interac*ng  with  Computers,  14(4):301–312,  2002.  

K.  Dey.  Understanding  and  using  context.  Personal  &  Ubiquitous  Compu*ng,  5(1):4–7,  2001.  

M.  Weiser.  The  computer  for  the  21st  century.  Human-­‐computer  interac*on:  toward  the  year  2000,  pages  933–940,  1995.  

P.  Persson  and  P.  Fagerberg.  Geonotes:  a  real-­‐use  study  of  a  public  loca*on-­‐aware  community  system.  Technical  Report,  2002.  

Russell  J.  A.  (1980).  A  circumplex  model  of  affect.  Journal  of  Personality  and  Social  Psychology,  39:1161–1178.  

34