Analysis of community-contributed space- and time-referenced data by example of Panoramio photos

Abstract

Space- and time-referenced data published on theWeb by general people can be viewed in a dualway: as independent spatio-temporal events andas trajectories of people in the geographical space.These two views suppose different approaches tothe analysis, which can yield different kinds of valu-able knowledge about places and about people. Wepresent several analysis methods corresponding tothese two views. The methods are suited to the largeamounts of the data.

1 Introduction

In the age of Web 2.0 more and more people publishvarious kinds of contents on the Web. Some kindsof contents have spatial and temporal references, forexample, the photos in Panoramio [9] and flickr [4]linked to the places where they were taken and sup-plied with the dates and times of the shots. Suchdata can serve as a source of knowledge about theplaces and about the interests, behaviors, and mo-bility of the people. The knowledge may be valu-able for local administrations, tourist services, ad-vertising agencies, and other organizations. How-ever, the data are not easy to analyze. One of theproblems is the huge number of entries, which callsfor scalable computational techniques. At the sametime, the involvement of a human analyst, who per-ceives spatial and temporal patterns and gives themmeaning, is essential.The community-contributed Web entries havingspatial and temporal references can be viewed,on the one hand, as independent spatio-temporalevents. On the other hand, the entries made by thesame person can be considered as a trajectory ofthis person in the geographical space, which tellssomething about the movement and behavior of thisperson. The whole dataset can be viewed as a setof trajectories of multiple people. These two views

suppose different approaches to the analysis, whichcan yield different kinds of knowledge. Another ex-ample of data that can be viewed in such a dual wayis data about mobile phone calls. We call this classof data event-based movement data.Analysis of event-based movement data is a rela-tively new research topic. A series of papers havebeen published by Girardin and co-authors (e.g. [5]and [6]). They analyze concentrations and move-ments of tourists at the scales of a city (e.g. Rome)and a geographical region (e.g. central Italy) usingFlickr photos and, in some studies, mobile phonecalls made by the tourists in the same time peri-ods. Concentrations are shown on heat maps pro-duced by dividing the area of interest by a rectan-gular grid and counting the number of photos in ev-ery grid cell. Interpolation techniques are used inorder to smooth the visualization. The movementsare visualized in an aggregated form by means offlow maps, where predefined places are connectedby lines with the widths proportional to the numbersof tourists that moved between the places. In [3],another research team uses geo-annotated Flickrphotos to find concentrations of activity and mostpopular places on Earth. For that they use a non-parametric MeanShift clustering algorithm. Theauthors have also used the temporal informationavailable from the photos to generate a few exam-ple maps showing the movements of photographers.The trajectories are represented by lines drawn onthe maps in a semi-transparent mode; however, nofurther analysis is made.

2 Our Approach and Methods

We take a systematic approach to the analysis ofevent-based movement data. We define possibletypes of analysis tasks related to the views of thedata as events and as trajectories. We distinguishspace-centered tasks, where the data are used tostudy the properties of the space and places, from

VMV 2009 M. Magnor, B. Rosenhahn, H. Theisel (Editors)

agent-centered tasks targeting at the properties andbehaviors of the people (in general, moving agents).For the tasks defined, we try to find or develop ap-propriate methods. Some of the methods are brieflypresented here.The example dataset we use for the presentationconsists of about 590 000 photos made on the terri-tory of Germany during the period from January 1,2005 till March 30, 2009. The methods we presentdo not take into account the contents of the photosbut only their spatial and temporal references.

2.1 Spatio-temporal aggregation of events

Aggregation helps an analyst to cope with largeamounts of data. Spatial and temporal aggregationof movement data viewed as independent events canbe done by means of database queries [1]. For thespatial aggregation, the territory is divided into suit-able compartments, for example, using a regulargrid. For the temporal aggregation, the time may betreated as linear or as cyclic and, respectively, di-vided into consecutive intervals or into intervals ofthe daily, weekly, or yearly cycle. The results of theaggregation are time series of counts related to thespatial compartments (number of photos, number ofdifferent people) and, possibly, other statistics suchas the mean number of photos per person.

Figure 1: A possible map display of aggregateddata.

Figure 1 gives an example of how aggregation re-sults can be visualized and explored. The data havebeen aggregated spatially by grid cells and tempo-rally by months, irrespective of the years. The mapfragment shows the south of Germany. The shadingof the cells portrays the total number of differentpeople who made their photos in these cells. Dy-namic filtering has been applied so that only thecells visited by at least 250 photographers are visi-ble. The diagrams represent the yearly variation ofthe number of photographers. The counts resulting

from the aggregation have been transformed into thedifferences from the local mean values normalizedby the standard deviations. The horizontal axis of adiagram represents 12 months from January to De-cember. The vertical dimension represents the de-viation from the local mean value. The color fillingserves to improve the visibility of the diagrams andthe differentiation between the positive and negativevalues (yellow and blue, respectively). The com-mon and distinct features of the seasonal variationsin different places are easily seen. For instance,a peculiarity can be noted in the area of Munich(marked in the figure): the number of photographersis very high from August till October, which may berelated to the famous Oktoberfest (beer festival).

2.2 Spatial clustering of events

Density map is a good technique to visualize con-centrations of events. A disadvantage of the grid-based approach described in [5][6] is that grids dontreflect the natural data distribution. Therefore, theresulting map may inaccurately show the places ofconcentration. Appropriate clustering methods areneeded to find the natural areas of high concentra-tion without imposing any artificial division of theterritory. We propose to use density-based cluster-ing algorithms (DBCA). The benefits are the fol-lowing: (1) regions with density below some thresh-old are counted as noise and wont be reflected invisualization. (2) The resulting clusters are not re-stricted to a specific shape or size and properly re-flect the natural distribution of the data.A frequently used approach to visualizing densitieson a map is color coding of the counts of points in-side areas, as in [5][6] and in Figure 1 (backgroundshading). A drawback is that the information aboutthe locations of the individual photos is lost. Wepropose to use the notion of influence function in-troduced in [7]. For every photo in a cluster, thecumulative sum of the influences of the other pho-tos is calculated using the Gaussian. Thus, everyphoto gets a weight based on the proximity of otherphotos around. When a density map is built, everyphoto gets its normalized color value. An exampleis given in Figure 2.

The map fragment portrays the central part ofBerlin. The densities of the photos are representedusing the color scale shown in the upper right cor-ner of the image. The idea is to express the popu-larity of the place (i.e. the density of photos in it) as

Figure 2: A density map showing the popular placesin Berlin.

the degree of heat. The right side of the color scalewith red-hot and white-hot shades corresponds tothe most popular places. Thus, the white spots inFigure 2 highlight the areas around the Bundestag,Brandenburg Gate, Potsdam Square, on the Mu-seum Island, and around the Alexanderplatz.

2.3 Spatiotemporal analysis of clusters ofevents

The Growth Ring Maps technique supports the ex-ploration of the frequencies and temporal patternsof events occurring in the same places. For thePanoramio dataset, we defined the significant placeson the basis of the density-based spatial clusteringof the events. For each place, we represent the pho-tos made in it by pixels placed around the centralpoint in an orbital layout [8] according to the timesof taking the photos: the earlier the photo was made,the closer the pixel is to the central point. Color-hue is used to map semantic properties of the eventsor places. Thus, seasonal differences in visitingthe places may be investigated by mapping the sea-sons to four distinct colors (winter-white, spring-green, summer-red, and autumn-orange). The re-sulting Growth Ring Maps show simultaneously theintensity of taking photos at different locations andthe seasonal differences. Figure 3 presents the re-sults for Berlin and Konstanz. While Berlin is vis-ited through the whole year, Konstanz appears as apopular place in the warm season, as indicated bythe Growth Rings with very little amounts of whitecolor.

Figure 3: Growth Ring Maps showing the seasonaldifferences of taking photos for Berlin (top) andKonstanz (bottom). The small amount of whitecolor in Konstanz indicates that warm months aremore popular at this region, whereas Berlin is vis-ited all over the year.

2.4 Analysis of flows

For this analysis, we build flow maps showingaggregated moves between places, i.e. how manypeople have moved from one place to another.There are two major differences from what isdescribed in [5] and [6]. First, our flow maps shownot only the amounts but also the directions ofthe movement by special half-arrow symbols. Itis easy to see whether the movements betweentwo locations are one- or two-directional and,in the latter case, whether one of the directionsprevails over the other. Second, we do not useany predefined places but divide the territory intoappropriate compartments on the basis of clusteringof the positions from the trajectories. By varyingthe clustering parameter (specifically, the desiredcluster radius), we can do the analysis at differentspatial scales. An example is presented in Figure4. We have taken the sequences of photos madein Berlin and surrounding area and divided theminto subsequences, or sessions, assuming that a

time interval of 8 hours or more between twophotos means the beginning of a new session.The sessions have been treated as trajectories.First, we tessellated the territory into bigger areasusing clustering with the desired radius 5km. Theupper map fragment shows the flows between theareas. By means of interactive filtering, we havehidden the flows corresponding to less than 10trajectories. We can see three disjoint regions ofmajor movement: the central part of Berlin (onthe east), Potsdam (on the southwest), and suburbson the west of the central Berlin. This means thatthe photo sessions are restricted in space. Thetime intervals between the photos taken in differentregions are usually longer than 8 hours.

Figure 4: The flows of the photographers in Berlinat two different spatial scales.

The lower map fragment represents the move-ments in the central part of Berlin aggregated at asmaller spatial scale (the cluster radius was 1km).By filtering, we have removed the flows corre-sponding to less than 40 trajectories. We see thatmajor movements occur along the street Unter denLinden. It is possible to see asymmetric two-directional movements. Thus, more photographersmove along Unter den Linden in the direction fromeast to west towards Brandenburg Gate than from

west to east. An opposite tendency can be seenon the southwest and south of this street, wherethe eastward movement Zoo - Potsdam Square -Checkpoint Charlie clearly prevails over the west-ward movement.

2.5 Analysis of interactions in space andtime

One of the possible analysis tasks is detection andinvestigation of collective movement patterns andsocial behaviors of the photographers. We supportthis task by tools for interaction analysis [2]. Weuse the term interaction for the kind of event whentwo or more people (in general, moving agents) arelocated in the same place at the same time. Wehave developed a computational method for extract-ing interactions from movement data. For each pairof trajectories, it tries to find respective positionssuch that the spatial and temporal distances betweenthem are within the given thresholds. In case of de-tecting such positions, the following positions of thetrajectories are checked. Spatial and temporal in-dexing of trajectory fragments is used for the sakeof efficiency.Of primary interest are interactions with long dura-tion (i.e. photographers spend significant time to-gether) and repeated interactions between the samephotographers. In order to detect these, we appliedthe tool to the 9884 trajectories having at least 10positions (i.e. photos made). We used the spatialthreshold of 1km and the time threshold of 2 hoursand got 3032 interactions that lasted minimum onehour. From the 9884 photographers, 2587 had someinteractions with others; however, only 1145 pho-tographers had two or more interactions with oth-ers and, among them, only 224 photographers hadrepeated interactions with some of the other pho-tographers. The maximum number of interactionsper photographer is 100; the number of repeated in-teractions between a pair of photographers rangesfrom 2 to 36.

Figure 5 demonstrates an example of multipleinteractions between two photographers. 14 inter-actions took place in Berlin and surroundings dur-ing the period from February 24, 2008 till July 19,2008. The interactions are represented on a map(top), time line display (middle), and in a space-time cube together with the trajectories of these twopeople (bottom). The trajectories are shown as blue

Figure 5: Multiple interactions of two photogra-phers.

and black lines; the interactions are marked in yel-low. It can be seen that the photographers do not al-ways move and make photos together but meet fromtime to time. Among the detected interactions, thereare cases when two photographers, apparently, trav-eled together from one city or area to another, e.g.from Leipzig to Hamburg. Specifically, there are27 cases when the maximum distance between theplaces of taking photos in one interaction is 20kmor more. Some of the long-distance interactions aredemonstrated in Figure 6.

Figure 6: Multiple interactions of two photogra-phers.

3 Conclusions

In most of the existing approaches to analysis ofmovement data it is assumed (in most cases im-plicitly) that the available position records repre-sent continuous trajectories and, hence, intermedi-ate positions can be obtained by means of interpola-tion between known positions. However, there aremany cases when the position records are tempo-rally sparse and irregular, which means that the datacannot be handled in this way. This kind of data canbe called event-based movement data. We suggest aset of visual analytics methods combining compu-tational techniques with interactive visual displaysto support the analysis of such data. The methodsare oriented to different classes of analysis tasks andenable the analyst to discover different kinds of pat-terns in the data, as is demonstrated by example ofPanoramio photos.

4 Acknowledgments

The work has been done within the research projectViAMoD Visual Spatiotemporal Pattern Analysisof Movement and Event Data, which is funded by

DFG Deutsche Forschungsgemeinschaft (GermanResearch Foundation) within the Priority ResearchProgramme Scalable Visual Analytics (SPP 1335).

References

[1] Andrienko, G., and Andrienko, N. Spatio-temporal aggregation for visual analysis ofmovements. In Proceedings of IEEE Sympo-sium on Visual Analytics Science and Tech-nology (VAST 2008), IEEE Computer SocietyPress, 51-58, 2008.

[2] Andrienko, N., Andrienko, G., Wachowicz,M., and Orellana, D. Uncovering Interac-tions between Moving Objects. In Cova, T.J.,Miller, H.J., Beard, K., Frank, A.U., Good-child, M.F. (Eds.): GIScience, 5th interna-tional conference, Proceedings, 16-26, 2008.

[3] Crandall, D., Backstrom, L., Huttenlocher, D.,and Kleinberg, J. Mapping the Worlds Pho-tos. International World Wide Web Confer-ence. 2009.

[4] Flickr. http://flickr.com[5] Girardin, F., Calabrese, F., Dal Fioro, F., Ratti,

C., Blat, J. Digital Footprinting: UncoveringTourists with User-Generated Content. Perva-sive Computing, IEEE, 7, 36-43, 2008.

[6] Girardin, F., Dal Fioro, F., Ratti, C., Blat,J. Understanding of Tourist Dynamics fromExplicitly Disclosed Location Information. In4th International Symposium on LBS andTelecartography, Hong-Kong, China, 2007.

[7] Hinneburg, E. and Keim, D. An efficientapproach to clustering in large multimediadatabases with noise. AAAI Press. 58-65,1998.

[8] Janetzko, H. Mansmann, F., Bak, P., andKeim, D.A Spatiotemporal analysis of SensorLogs using Growth Ring Maps. IEEE Trans-actions of Computer Graphics and Visualiza-tion,2009 (accepted).

[9] Panoramio. http://www.panoramio.com