what's in a place? adventures with location-aware media
DESCRIPTION
Can we automatically create an “attraction map” of the world from Flickr geotagged images and their associated tags? We performed an analysis of Flickr data and developed a visualization technique called Tag Maps to do exactly that. Using the analysis and the Tag Maps visualization, we created an exploration tool called World Explorer that allows one to, well, explore the world like never before. The idea behind the data analysis is simple: by taking a photo, photographers essentially express their interest in a particular place, and implicitly “vote” in favor of that location. This gives us a set of highly representative tags associated with each map location. The World Explorer visualization is facilitated by placing these representative tags on a map (“a Tag Map”). We augment the Tag Map with photos that represent each tag at its specific location. Together, World Explorer effectively provides a sense of the important concepts and attractions embodied in each map area and zoom level, and allows users – tourists planning a trip, virtual world-discoverers or just some bored individuals – to explore the world via photos. I’ll also give a brief demo and overview of Zurfer, a novel mobile phone context-aware software prototype that enables access to images on the go. It utilizes the channel metaphor to give users contextual access to media of interest according to key dimensions: spatial, social, and topical. Zurfer attempts to be playful and simple to use, yet provide powerful and comprehensive media access. A temporally-driven sorting scheme for media items allows quick and easy access to items of interest in any dimension. For novice users, and more complicated tasks, we extend the application incorporating keyword search to deliver the long tail of media and images. http://www.rahulnair.net/blog/2007/06/07/talking-at-ricoh-what%e2%80%99s-in-a-place/TRANSCRIPT
1yahooresearchberkeley.comRahul Nair
What’s in a place?Adventures with Location-Aware Media
Rahul Nair
Yahoo! Research Berkeley
2yahooresearchberkeley.comRahul Nair
Flickr “geotagged”
20+ million images
Can we do better?
3yahooresearchberkeley.comRahul Nair
Talk Outline
• Extracting information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
4yahooresearchberkeley.comRahul Nair
Attraction Map of Paris
Stanley Milgram, 1976. Psychological Maps of Paris
5yahooresearchberkeley.comRahul Nair
Attraction Map of Paris
Y!RB, 2007.
6yahooresearchberkeley.comRahul Nair
Location-driven Modeling
• Derive meaningful data about map regions
• E.g., representative tags, photos
7yahooresearchberkeley.comRahul Nair
Data Description
8yahooresearchberkeley.comRahul Nair
Issues
• Sparse data set
• Photographer bias
– In location
– In tags
• Incorrect data
9yahooresearchberkeley.comRahul Nair
Heuristics
• Number of photographs denotes the “importance” of a location
• Users will use a common subset of tags to describe objects/locations
• Concentrated tag usage indicates descriptiveness
10yahooresearchberkeley.comRahul Nair
Algorithm
• Clustering: k-Means, get set of k clusters
• “Document” C is bag of all tags in cluster
• For each tag in C calculate:
– TF = |P(C,t)|
– IDF = |P(R)| / |P(R, t)|
– UF = |U(C,t)|/|U(C)|
11yahooresearchberkeley.comRahul Nair
Scoring
• Score (t) = TF * IDF * UF
• Threshold values
– 30+ photographs
– Minimum 3 users
– Score > 1
• Final dataset: (tag, score, latitude, longitude)
12yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
13
DEMO
14yahooresearchberkeley.comRahul Nair
Precomputation
• Divide the world into equal sized non-overlapping tiles
• Compute and store the tags for each tile
• Repeat for different zoom
levels
15yahooresearchberkeley.comRahul Nair
Retrieval
• Find the tile level closest in size to the request area
• Select the tiles that fully cover the request area
• Return the tags that fall within the request area
16yahooresearchberkeley.comRahul Nair
User Study
10 subjects
• 6 female, 4 male
• Ages 20-60
• Varying technical knowledge
• No geotagged photos of their own
17yahooresearchberkeley.comRahul Nair
Experiment tasks
• Vacation recap
• San Francisco tour
• Explore a new city
18yahooresearchberkeley.comRahul Nair
Recall
Reminded the subject about locations
• “It brings out memories”
• “Oh my God! This place has the best restaurants”
• “We wanted to see the Polynesian Cultural Center"
19yahooresearchberkeley.comRahul Nair
Discovery
Participants discovered previously unknown locations and events
– “I’ve never heard of this festival”
– “There is car racing which I'd probably go see”
20yahooresearchberkeley.comRahul Nair
Needle & Haystack
• Excellent visualization of the Haystack
• Hard to find specific information
– “Where was Culver City again?”
• No way to search
– “I guess what I’m looking for are bull fighting pictures”
21yahooresearchberkeley.comRahul Nair
Other Responses
• Gets the “vibe” of a place
• Share with other people
• Tags did not always match the mental model of a location
• Wanted more tags
• Want more info about tags
22yahooresearchberkeley.comRahul Nair
Conclusions
• Extract meaningful aggregate information from georeferenced data
• Allows users to explore locations in a new way
• Users like using the overview but also want the ability to search
23yahooresearchberkeley.comRahul Nair
Future work
• Adding search capability
• Show photos in places with no tags
• Differentiate locations and events
• Apply to other types of georeferenced data
24yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
25yahooresearchberkeley.comRahul Nair
Current Mobile Experience
• Difficult to share (or even save!)
• Hard to find
– No context
– No semantic information
Current mobile experience?
26yahooresearchberkeley.comRahul Nair
Current Mobile Experience
• Difficult to share (or even save!)
• Hard to find
– No context
– No semantic information
27yahooresearchberkeley.comRahul Nair
ZoneTag Experience
• 2-click upload (same key!)
• Photo uploaded with location and time metadata
28yahooresearchberkeley.comRahul Nair
Where does location come from?
• Bluetooth GPS (when available)
• User-contributed cell tower mapping
29yahooresearchberkeley.comRahul Nair
ZoneTag Experience
• Tagging made easy
– Tag/annotate your photos from the phone
30yahooresearchberkeley.comRahul Nair
Where do tags come from?
• Tags I used in this context (`home’)
• Tags my friends used in this context
• Tags other people used in this context (‘Ricoh’, ‘California Research Center’)
– E.g., TagMaps data
31yahooresearchberkeley.comRahul Nair
Where do tags come from?
• Stuff around you:
– Yahoo! Local
– Upcoming.org
• Stuff from you (any RSS 2.0 feed):
– Calendar
– Favorite hangouts (Wayfaring, Plazes, Socialight)
32yahooresearchberkeley.comRahul Nair
Example of Tagging
34yahooresearchberkeley.comRahul Nair
Talk Outline
• Mining information from geo-tagged photos
• World Explorer: Visualization
• ZoneTag: Creation
• Zurfer: Consumption
35yahooresearchberkeley.comRahul Nair
Flickr “geotagged”
20+ million images
Can we do mobile?
36yahooresearchberkeley.comRahul Nair
Strengths of Mobile
• Personal
• Easy to access
• Networked
• Context aware
The ultimate photo wallet
37yahooresearchberkeley.comRahul Nair
Design Goals
• Engagement and Discovery
– Spatial
– Social
• Allow customization
• Complete access to your photos
• Search and Filtering capability
38yahooresearchberkeley.comRahul Nair
Channel Metaphor
• Each row is a single channel
• Navigate using 4 way
– Left & right to browse channel (infinite scroll)
– Up & down to change channels
39yahooresearchberkeley.comRahul Nair
Detail view
• Enlarge photo
– Scroll through channel
• Photo details
• Add comments
• Add to favourites
40yahooresearchberkeley.comRahul Nair
Nearby photos channel
• Photos from the users current location
• Local Highlights
• My Nearby
41yahooresearchberkeley.comRahul Nair
Social Channels
• Contacts photos
– Expanded view
• Recent Activity
42yahooresearchberkeley.comRahul Nair
My Stuff Channel
• My Photos
– By Location
– By Tag
• My Photo Wallet
• My Favourites
43yahooresearchberkeley.comRahul Nair
Custom Channels
• Users can create channels to match their interests
– Tags
– Groups
– Location aware (dogs near me)
44yahooresearchberkeley.comRahul Nair
Search and Filter
45yahooresearchberkeley.comRahul Nair
Try it out
http://zurfer.research.yahoo.com
46yahooresearchberkeley.comRahul Nair
Conclusions
• It is possible to extract information from georeferenced media
• Users like browsing the extracted data
• It can help users tag new media
• We hope it helps them browse on a mobile device
47
Questions?
Rahul [email protected]
http://tagmaps.research.yahoo.comhttp://zonetag.research.yahoo.com http://zurfer.research.yahoo.com