European Project No: FP7-614154 Brazilian Project Nº: CNPq-490084/2013-3

Project Acronym: RESCUER

Project Title: Reliable and Smart Crowdsourcing Solution for Emergency and Crisis Management

Instrument: Collaborative Project

European Call Identifier: FP7-ICT-2013-EU-Brazil Brazilian Call Identifier: MCTI/CNPq 13/2012

Deliverable D4.2.2 Visualisation Mechanisms for Emergency Coordination 2

Due date of deliverable: PM16

Actual submission date: January 31, 2015

Start date of the project: Europe: October 1, 2013 | Brazil: February 1, 2014 Duration: 30 months Organization name of lead contractor for this deliverable: VOMATEC

Dissemination level PU Public PP Restricted to other program participants (including Commission Services) RE Restricted to a group specified by the consortium (including Commission Services) CO Confidential, only for members of the consortium (including Commission Services)

Executive Summary Visualisation Mechanisms for Emergency Coordination 2

This document together with the respective implemented prototypes compose deliverable D4.2.2 (Visualisation Mechanisms for Emergency Coordination 2) of project FP7-614154 | CNPq-490084/2013-3 (RESCUER), a Collaborative Project supported by the European Commission and MCTI/CNPq. Full information on this project is available online at http://www.rescuer-project.org.

Deliverable D4.2.2 provides the results of Task 4.2 (Visualisation Mechanisms for Emergency Coordination) for the second project iteration. This deliverable shows findings after investigating the best way to present data for emergency coordination, and improvements in the emergency mapping visualisations provided in the first iteration. The purpose of this task is to accomplish a clear and efficient communication with the command and control centre, regarding information provided through crowdsourcing.

This deliverable will be extended in the next project iteration to give rise to D4.2.3 (Visualisation Mechanisms for Emergency Coordination 3). Through this iterative process, more features will be implemented and tested to finally get an optimized and comprehensive solution. List of Authors

Silas Graffy – VOMATEC Matthias Breyer – VOMATEC Laia G. Pedraza – VOMATEC Paulo Junior – UFBA Renato Novais - UFBA

List of Internal Reviewers

Juan Torres – UPM Jose F. R. Junior – USP Vaninha Vieira – UFBA

2

Contents 1. Introduction ..................................................................................................................................... 5

1.1. Purpose ........................................................................................................................................ 5

1.2. Change Log .................................................................................................................................. 5

1.3. Partner’s Roles and Contributions............................................................................................... 5

1.4. Document overview .................................................................................................................... 6

2. Visualisation Concepts ..................................................................................................................... 7

2.1. Fundamentals in Information Visualisation ................................................................................ 7

2.1.1. Visual Features ........................................................................................................................ 8

2.1.2. Interaction Concepts in Visualisations .................................................................................... 8

2.2. Visualisation Techniques in Emergency Management Tools ...................................................... 9

2.2.1. Related Work on Information Visualisation in Crisis and Emergency Management ............ 10

2.2.2. Remarks about Related Work ............................................................................................... 18

2.3. Visualisation Concepts for Video Analysis Results .................................................................... 18

2.3.1. Background on Stream Processing ........................................................................................ 19

2.3.2. Current Research on Stream Visualisation ............................................................................ 19

2.3.3. Design based on Metaphors and Analogies .......................................................................... 20

2.3.4. Remarks ................................................................................................................................. 20

2.3.5. Features Extraction, Distance Functions, and Metric Space ................................................. 21

2.3.6. Novelty Detection over Data Streams ................................................................................... 22

3. First Prototype of the Visualisation Mechanisms .......................................................................... 23

3.1. Overview .................................................................................................................................... 23

3.2. Layers ......................................................................................................................................... 23

3.3. Detail information ..................................................................................................................... 25

3.4. Additional Information .............................................................................................................. 25

3.5. Technical Background and Requirements ................................................................................. 26

4. Second Prototype of the Visualisation Mechanisms ..................................................................... 27

4.1. Overview .................................................................................................................................... 27

4.2. Improvements for the Emergency Mapping ............................................................................ 27

4.3. Workforces Map Layer in Map View ......................................................................................... 33

4.4. Incident Detail ........................................................................................................................... 34

4.5. Workforces and Tasks Coordination ......................................................................................... 35 3

4.6. Emergency Overview ................................................................................................................. 36

4.7. Technical Background and Requirements ................................................................................. 37

5. Conclusions .................................................................................................................................... 38

References ............................................................................................................................................. 39

4

1. Introduction

1.1. Purpose

An Emergency command and control centre supporting decisions making should transmit information in a clear and efficient way to decrease the time users need to process or find information. RESCUER aims to offer a Real Time Dashboard with visualisation metaphors that accomplish these characteristics.

Information visualisation is done in many ways and the work here is to identify the best options for each kind of information. A literature review was carried out to know what is already done in current systems and how they represent information. An aspect to take into account is the user interaction with the objects of the emergency command and control centre. It is useful to see how the same information is presented, to be analysed and compared for different purposes.

RESCUER system also works with image and video data. The analysis results of these elements have to be shown in the command and control centre. For the streaming visualization, it has been required a little bit more of literature review since it needs different techniques in order to get the results out of it. Therefore, it has a special mention in this document.

1.2. Change Log

This deliverable is a living document. It extends the contents of the first iteration (D4.2.1) by adding new contents. It updates sections with information that was not available at the time of the first iteration and improves the overall deliverable.

New research has been performed and the new found results are presented in this document, enriching the section about the visualisation concepts. The RESCUER second iteration focuses on group-target follow up interaction and coordination among workforces, therefore the results about this new investigation has also been added.

The result of the first iteration of Task 4.4, the implementation of the Emergency Response Tookit, has influenced the second iteration of this task. The new prototype has more functionalities and more contents. Therefore, there are more possibilities to show the visualisation concepts and more needs for better visualisations came up.

A new chapter has been created to present the prototype developed in this second iteration, which includes the improvements made for the visualisation of the first prototype and the visualisations applied to support the coordination among workforces.

1.3. Partner’s Roles and Contributions

VOMATEC coordinated the elaboration of this document and also contributed with the description of the visualisation concepts, and the implementation of the visualisation mechanisms prototype. UFBA conducted a literature review on information visualisation in emergency management tools and will contribute from now on with the implementation of the prototype.

5

1.4. Document overview

The remainder of this document is structured as follows. • Chapter 2 describes results of the research about information visualisation, the main

features and interactions that should be considered. There is a special subsection for the visualization of video analysis results.

• Chapter 3 introduces the first iteration of the visualisation prototype for RESCUER, focused on emergency mapping.

• Chapter 4 introduces the second iteration of the visualisation prototype for RESCUER, focused on coordination of the workforces.

• Chapter 5 defines the ongoing steps to be accomplished in the next iteraction. • Chapter 6 summarizes the conclusions obtained with this task.

6

2. Visualisation Concepts It has been noted, in recent years, a consensus that information visualisation can play a critical

role in analysis and decision making processes, as can be seen in major events and vehicles in the area, such as the IEEE Information Visualisation (InfoVis) conference. Visualisation can aid the discovery of different patterns such as clusters, associations, relationships, and trends. This area of research, formally called Visual Analytics (also, information visualisation or visual data analysis) is based on the fact that interactive visual environments can match human cognitive capabilities with high-performance computing, increasing the speed and accuracy of pattern discovery by experts [33]. In 2009, the journal Business Week [28] categorically stated that “There are real implications for business in information visualisation”, adding that such a practice is a way to extract meaning from the increasing flow of information, providing an antidote to the so-called analytical paralysis.

First of all, the fundamentals in information visualisation are discussed. After this results, there are described the results of the literature review made in order to find interesting visualisation techniques for emergency management tools. These results will allow us to have a better picture of what is out there and what can interest us when designing the RESCUER ermergency response toolkit. Finally, the video analsys result has a special mention, since it needs more technical solutions and a deeper literature review has been made.

2.1. Fundamentals in Information Visualisation

Information Visualisation comprises the representation of data in a way that the user can perceive it easily, identify and extract the important information artefacts, which have or should have impacts on his/her behaviour and decisions. Therefore, the field of information visualisation is interdisciplinary, mainly comprised by psychological aspects like human perception, cognitive models, and spatial mental models; data analysis like relations, pattern identification and matching; visualisation aspects like layouts, colour schemes, and interactions.

According to the general process model of information visualisation [16], the development of a visualisation concept follows the questions “What raw data have to be visualised?”, “Which inherent semantic structure is given in the data?” and “How should the data and its structure be transferred to a visual representation?”. This leads to the following process [35]:

1. Raw data analysis

a) data types

b) inherent semantic structure

c) data transformation

2. Define visual mapping for representation

a) for data items according to data type

b) for semantic structure and relations

3. Apply view transformation and interaction mechanisms

7

2.1.1. Visual Features

According to [5, 29] visual features describe visual variables like position, shape, orientation, colour, which can be utilized to portray and thus transport information to the user. Bertin [5] differentiates visual features in the following perception characteristic:

- Associative perception: degree of coverage/overlay of one visual feature over another one; - Selective perception: describes how intuitive one object is distinguishable from others; - Ordered perception: reflects the perceived (not mathematical) order; - Quantitative perception: qualifies the ordered perception (limitation).

Mackinlay [24] extended this to a model of which visual features should be used based on the type of information given (quantitative, ordinal, nominal). His model describes a ranking of visual feature perception with relation to the data type, shown in Figure 1.

Figure 1: Ranking of visual feature perception with relation to data type [24]

2.1.2. Interaction Concepts in Visualisations

According to Shneiderman [30, 31], the most important interaction fundamental seems to be the “direct manipulation”. To ensure an easy-to-use and thus easy-to-interpret visualisation, the user should feel like interacting with a real world object. To concretize this concept, several interaction mechanisms evolved [37, 17, 21, 45, 46]:

- Pan&Zoom - Overview&Details - Focus&Context - Projection - Filtering - Distortion - Link & Brush - Semantic zoom / semantic level-of-detail

8

- Graphical zoom - Visual clustering - Details-on-demand - Fisheye views

On an upper level, Shneiderman’s Visual Information Seeking Mantra [57] should be followed, describing the visual workflow from overview first, zoom and filter, details on demand.

2.2. Visualisation Techniques in Emergency Management Tools

To understand the visualisation techniques most used by current emergency management tools, it was conducted a systematic mapping study. This mapping allows analysing how information visualisation is structured in the emergency and crisis management context. First, we defined a protocol (see Table 1) with the research question, search string, search digital databases, and inclusion and exclusion criteria.

Table 1 Brief description of the Mapping Study protocol

Research question Which visual metaphors are being used to manage crises and emergencies?

Search string ("crisis management" OR "disaster management" OR "emergency management") AND ("visualization" OR "visual" OR "visualisation")

Digital databases Science Direct DBLP ACM IEEE Scopus Engineering Village (Compendex) Web of Science

Inclusion criteria Presents at least one visual metaphor Crisis and emergency context Language in English

Exclusion criteria Gray-literature Secondary studies Year < 2001 Works without abstract No available full text

Many papers address crisis and emergency in a general context. We found many works in the

following areas, not related to the RESCUER project goals: economic crisis, hospital emergency and so on. As they are not well aligned with the goals of the project, we excluded these works from the

9

review. On the other hand, we included papers related to catastrophes specific areas, like flood, fire, tsunami, hurricanes, and so on.

The results of this mapping will be presented in the final version of this deliverable. In the current state of the research, the papers presented in section 2.2.1 show the trending features and visualizations in systems applied to emergency management.

2.2.1. Related Work on Information Visualisation in Crisis and Emergency Management

The work presented in [47] is focused on crisis (emergency) management with strong theoretical background about the subject. It uses the icon-based and geometric projection paradigms that are plotted on maps for collaborative decision-making. Figure 2 explains this concept. There are two maps: one is private and the other one is public (collaborative). In general, the groups work (draw sketches, make annotations, etc.) in the private map then diffuse results to the shared one. Annotations and sketches are all colour-coded according to the role of the creator, as indicated by the role colour legend above the public map. When an item in the sorting table is clicked (second line in the figure), its corresponding representations are highlighted in the aggregation chart (the top red dot on the second column), in the timeline (the single red dot in the middle line) and in the public map (the focused yellow annotation). By seeing these different symbols of the same object in different visualisation tools, the user can examine the same piece of information from different perspectives and within different contexts.

Figure 2: User interface of web-based prototype [47]

10

In [48], the focus is a visualisation technique described in smartphones; the aim is to reduce the

visual clutter caused in plotting large data on a map. The technique works basically with 3 variables: coverage, overlapping and number of clusters (the filter bars in yellow as illustrated in Figure 3). The user can modify in real-time the coverage and the overlapping thresholds and the number of clusters. The algorithm keeps the coverage and overlapping percentages below the respective thresholds by scaling down the cluster sizes and if it is necessary some clusters are colored in grey degree. Higher thresholds mean higher clutter (see Figure 3).

Figure 3: Cluster visualization [48]

In [49], authors use Twitter micro blogging messages as data source of the plots on the map. The idea is to analyse public behaviour during a crisis through the use of visualisation of the posts metadata (spatiotemporal) on Twitter. Visualisation for spatiotemporal social media data is shown in Figure 4 (left). A hexagon represents the spatial (position) and temporal (colour) information of a tweet. Hurricane evacuation map is shown in Figure 4 (right). Residents in Zone A (red) faced the highest risk of flooding; Zone B (yellow) and Zone C (green) are moderate and low respectively. The objective of the figure is to demonstrate that many people still remained at home one day after the mandatory evacuation order in 10/29/2012 of Red Zone (due to red and orange hexagon plotted in the map).

In [50] a system that uses Twitter data to generate visualizations is presented. It can give emergency management agents geographically grounded situational awareness. An important result presented is a survey, with people from the International Association of Emergency Managers and U.S. Department of Homeland Security1, were they identify features that would be useful for this kind of system. Time graphs, keyword clouds and clustering tools are among them. Figure 5 shows a mock view of the system used in the survey.

1 http://www.firstresponder.gov/SitePages/HomePage/about.aspx 11

The keyword clouds allow the user to get more information, not only the meaning of the word

but its relevance in the context given by the other.

Figure 4: Visualisation for spatiotemporal social media data [49]

Figure 5 - Sense Place 2 mockup for survey

12

In [51] and [52], computers in table format (the tabletops) are used. In general, they have greater

processing capacity than simple desktops and due to its horizontal layout, screen size and screen resolution they improve visualisation and interaction with a map and plotted data. As shown in Figure 6, tabletops also permit the interaction with other devices like smartphones.

Figure 6: Interface between smartphone and tabletop [51]

In [53], authors propose a solution to help managing crises, specifically cases of flooding. The solution consists in a mobile application, which uses the Twitter social network for information dissemination when a crisis outbreak is reported. The application offers a visualisation tool that uses spatiotemporal map and markers, which can verify an animation in space and time of the event along with the report made by the user. As shown in Figure 7, an event is reported through the tool (a); (b) shows a screen of animation events reported by citizens and radar data over the last 10 hours; after selecting the preview option in the tool, you can view an animation on space and time along with the events reported by the user on the presented area (c).

In [54], there is a solution for evacuation situations in case of natural disasters (tsunami, flood, etc.) intended to prevent disasters. The application assists normal and blind people through guidance on escape routes. Basically, it consists in alerts for the user. The user registers in the system and sees records of hazardous areas for the occurrence of disaster. Based on the user's location obtained by GPS, it is identified if the user is in an area of risk. If yes, he/she is alerted by a notification server through both audio and video. In this case, the user receives guidance about the way to escape from the danger zone, visualized on a 2D map. Figure 8 shows the application interfaces where the user can start the service notification (a), and receive alerts when he/she is in a risk area (b).

13

Figure 7: (a) An event reporting tool on Windows Phone 7; (b) A real-time spatiotemporal visualisation tool on the smartphone; (c) Spatiotemporal events visualisation within Champaign,

Illinois on the smartphone during a experimental test [53]

Figure 8: (a) Starting service to notify automatic warning; (b) The notification status [54]

Figure 9 shows the service that checks whether the user is currently in any area affected by a potential disaster in a situation of evacuation. This service sends the user's location to the DMS (Disaster Management Server) when the user changes its location. The DMS is responsible for tracking user data.

14

Figure 9: Tracking users and available rescue team information [54]

As shown in Figure 10, with the data stored in the DMS, the application shows the shortest path between the current location and the area under the shelter (a); the application interface provides a menu with two options: one option to see a guidance message for evacuation of the area at risk (Figure 10b) and another one (Figure 10c) provides the direction to go to the shelter via audio.

Figure 10: (a) Application shows the shortest path between current location and shelter; (b) The message and audio direction options in menu button; (c) The visual message [54]

Figure 11 shows how the application registered the occurrence of a possible disaster, as well as the message that should be reported to the user who is in the area of risk. All these records are stored in the DMS.

15

Figure 11: Updating a region of probable disaster affected area along with disaster name and specific warning [54]

The project presented in [55] is in the context of large-scale rescue operations, which uses multimedia communication and information services. Figure 12 shows the online mapping system that allows visualisation of a location or route. The SHARE system integrates the mapping component (MAP3D) by providing appropriate data and functions to be understood and manipulated by the user (a). To achieve this, the application uses maps in 2D and 3D perspective (b).

Figure 12: (a) MAP3D application with street information, sections and resources; (b) 2D and 3D view of landmark (stadium) [55]

Finally, in [56], authors propose a system to support disaster management using Geographic Information System (GIS) technology. A representative of the support service to people who are in a zone of occurrence of a disaster (in this case the municipal employee) is responsible for registering in the system, photos of the disaster with a phone or smartphone. Figure 13 shows how the GIS

16

technology allows images of the disaster site, the photos taken by the municipal employee, can be viewed on the map, in this case, the areas of the map that have a "star". This enables more realistic view of the scene of occurrence.

Figure 13: GIS digital mapping technology [56]

Figure 14 shows the PIECES system [58], which is a flexible Precision Information Environment capable to display data coming from a variety of sources. It helps the coordination of workforces by giving the command and control centre the ability to see the workforces in the field and give them instructions. It shows nearby places like hospitals, where the workforces can be directed to send the injured people. It also provides interaction with them through a mobile application, allowing the command and control centre to send images and videos, to give the workforces information they can use to complete their tasks.

Figure 14: PIECES system with multiple information displayed

17

2.2.2. Remarks about Related Work

A large number of papers have been analysed for the literature review. The results confirm the first impressions about the use of maps and icon-based paradigms. These two are frequently used together, and give the user information about “what is happening” and “where is happening”. If the icons are intuitive this can greatly improve the understanding of the visual scene by the user. It was also observed that in most situations involving emergency crisis management, both sides are involved: the population and support personnel [54][56]. There is the need for cooperation between both [55].

Systems that are designed to support emergency response are often used by more than one user in command and control centres. Given the relevance of collaboration in these situations, it is noted that the choice of suitable devices for this interaction is required, e.g. phones, smartphones, touchscreen to the command centre, and therefore, adapting environment information viewing for each device. Features like annotations and drawing [47] are usually present in this case. Also, these systems should provide features that allow the command and control centre users to communicate with the workforces in the area of incident.

The more relevant concepts found in visualization are the following: • Use of icons and maps to give users geographic awareness of the incident [47][58]; • Heat maps and geometric shapes, together with colours, to represent areas of interest,

incident impact area, density (number of people in the area, concentration of toxins in the area, etc.);

• When using the map, it is important to note that 3D visualisation is not only important to aid orientation, for example, in cases of evacuation, but also enables to explore more details, which can not be seen in a 2D perspective [55];

• Use of time lines to give user temporal awareness of the emergency evolution.

In case of trending features, the results show that filters and/or queries are often applied in case of systems that present multiple types of information. In maps, the use of multiple layers is useful to show/hide information according to the needs of the users. Web systems, or those who get information from a web service, are used when multiple users should be using the system at the same time. The ability to make annotations, drawings and add detailed information in the map, or to the system is considered useful by emergency management agents.

Most of the described systems present evaluation results or surveys with emergency management specialists. These results are a guide in order to analyse the features of RESCUER Emergency Response Toolkit, e.g. whether the features are useful or not.

2.3. Visualisation Concepts for Video Analysis Results

For the crisis situations predicted for the RESCUER solution, one major characteristic is the continuous arrival of new information in the form of video, image, and text along time. This information is of large scale and of urgent interpretation. These characteristics pose great problems: first, it is not possible to store all the information because of its intense flow and huge storage needs;

18

second, the information must be interpreted on the fly despite its challenging computational cost. This problem is well-known from the computer science literature and referenced as “stream processing”, which comprehend several lines of work like mining streams, summarizing streams, and visualising streams, to name a few.

In this context, the Emergency Response Toolkit (ERTK), more specifically its visualisation solution, must employ stream visualisation techniques in order to satisfy the aforementioned needs. In this section, we review techniques and previous ideas of how to bring stream visualisation in the context of the ERTK; of high-priority concern, we describe how such techniques may adhere to a geographic referential – a background map, as it is the basic design element of the toolkit.

2.3.1. Background on Stream Processing

In contemporaneous applications, whose usage is booming out, as network monitoring, telecommunications management, flow analysis, Web clicks analysis, tracking sensor networks, maintenance of production lines, and understanding of economic systems [8] the data have the format of streams – unidirectional continuous flows of data. In these applications, there is a need to perform analysis at the same speed at which the data are collected; in computer networks, for example, it is not interesting to store the data so to detect, only subsequently, a network. In the area of databases, researchs have focused on processing, analysing, and managing streams [32]. The topic of streams is recurrent in many disciplines of computer science, such as databases [4], machine learning [15], and information visualisation [9] [26] [38].

2.3.2. Current Research on Stream Visualisation

In this context, many works were devoted to stream visualisation. In the area of textual streams, Alsakran et al. [3] present the system Streamit, which enables the understanding of textual data without prior knowledge of the information; the proposal uses a technique based on forces to design and group continuously arriving text using similarity. The StreamSqueeze system [25], proposed by Mansmann et al., considers how recent the data are in order to define its importance, which is encoded in the size and detail of graphical items according to a dynamic list arrangement. The TweetProbe proposed by Kang et al. [20] uses animation and various projection techniques to elucidate the contents of the instant messaging data. In another study, Zaixian et al. [39] introduce a data-driven framework to combine and condense time windows having no or little change; this way, only the most significant changes are shown to users.

Due to being relatively recent in visualisation, the vocabulary related to stream visualisation has not been standardized. Norton et al. [26] use the term “streaming graphics”; Fry [13] proposes the term “organic information design”; there is also the term “dynamic visualisation” [9][41]. Besides, there are plenty of work in visualisation of temporal data [2]; dynamic data updated over time.

Krstajic et al. [22] present the technique CloudLines which combines graphical lines built while new data items arrive; the technique is interesting since it describes the need to emphasize the new information and because it uses a decay function to express distortion over time. In the study by Norton et al. [26], the authors propose a technique designed to refresh the visualisation in the

19

immediate moment of incoming information. They discuss design issues such as the fact of the timescale be discrete or continuous, update strategies, and comparisons between real-time and replay presentation. However, the work is limited to graphical processing restrictions and respective solutions rather than metaphors and design guidelines.

2.3.3. Design based on Metaphors and Analogies

There are several studies that propose the use of metaphors in visualisation and user-computer interaction. There are metaphors based on lenses such as the Fisheye Views classical technique [14], the Zoomable interfaces [27], the Tool-glasses and Magic Lenses techniques [6], the Magic Lenses’s variation Colour Lenses [10], and Excentric Labels [11]. There are also metaphors based on geological sedimentation, like the Bubble-t [36] [18]; in this proposal, developed for the presidential elections in France, there is a bar graph for each candidate and, as an incoming electronic message (tweet) mentions a given candidate, a new graphic element gravitationally falls within the respective graph bar where it accumulates. Using a model of forces, the work of Whisper [7] presents real-time broadcasting of electronic messages according to their topics; the proposal uses an arrangement of circular layers so that the inner layers progressively comprise older graphics. The work of Viegas et al. [34] uses a presentation similar to geological layers; the authors compress the information so as to define layers as can be observed in rocks. In turn, the work of Lin and Vuillemot [23] uses spirals drawn as complex arrangements of flower petals to represent the flow of data also stemmed from instant messaging. Finally, the work of Huron et al. [19] describes a framework for the development of visualisation techniques using the concept of sedimentation. This work consists of an extensive programming library that supports several dynamic arrangements using force simulations and interaction.

2.3.4. Remarks

The works on stream visualisation are mainly based on metaphors and analogies to physical systems. Although there are many works found in the literature, a lot is yet to be done. In the context of the RESCUER project, these works must be adapted considering three aspects:

1) The majority of what has been done until now is based on textual data; meanwhile, RESCUER will use video and image data – as well as text. Therefore, the techniques must be adapted to handle these kind of data; this adaptation can be done with features extraction techniques, as explained in section 2.3.5;

2) Existing techniques were not designed having a geographical reference in mind, therefore they must be redesigned to be presented on top of the ERTK visualisation, possibly by means of summarizing icons; it is worth noticing that the ERTK is being designed so that the user can easily switch between visual elements in order to focus on different aspects of the data;

3) Although stream visualisation can be summarized as icons and selectively displayed following user interaction, the stream data can be better observed if details-on-demand are available under user request. In the ERTK visualisation, one can accomplish it by using the lenses-based metaphors presented in section 2.3.3.

20

2.3.5. Features Extraction, Distance Functions, and Metric Space

Complex data – video, audio, image, and text – pose a classical problem in computer science. The data are complex since there is not a straightforward manner to extract valuable information from them. Since it is a long-term problem, it has a well-defined field of research, known as content-based retrieval, which has converged to sets of techniques that allow representing complex data according to numerical representations, to index the data, and to retrieve the data following similarity queries. Although stream techniques was originally designed for textual data, it can be used with complex data, as demanded by the RESCUER project. To adapt the data some techniques can be applied, such as features extraction.

Features extraction

The features extraction corresponds to processing the data so that a vector of n representative numbers is extracted from them. Classical examples, in the case of images, are the colour histogram [12] and the coefficients achieved with the Fourier transform [40]; both of them define a vector of numbers. Along this text, we refer to features extraction in general as a function f:D D, where D is a domain of specific data, and D ⊂ Rn is a n-dimensional features space, e.g. images where n=2 or videos where n=3.

Distance function

The second step in order to define a metric space is to establish a similarity measure, or distance function, among the vectors of numbers extracted from the data objects. A trivial way to do this is to consider each numerical feature as a n-dimensional coordinate and calculate the Euclidian distance between the vectors. Other examples of distance functions are the City Block and the Minkovisk distances [1].

Metric space

Once feature vectors and a distance function are specified, a metric space is established. A metric space refers to a set in which the notion of distance between its elements is well-defined. Formally, a metric space is a pair M = <D,Δ>, where D is the domain of the elements to be indexed and Δ:D xD R+ is a function that associates a distance to any two elements o_i, o_j ∈ D.

Applicability

Content-based retrieval can be used to computationally process many kinds of data that natively are not “computer-friendly”. Such concepts are used in database applications, data mining, machine learning, and visual data analysis. In the RESCUER project, content-based retrieval is to be used to assist in the definition of stream visualisation – as explained in sections 2.3.1 through 2.3.4 and, furthermore, in stream processing techniques that may assist the visualisation process – as will be explained in section 2.3.6. To this end, video, audio, image, and text are to be considered according to their features. For example, in the case of fire detection in images and videos, simple features as

21

the amount of red pixels can be used; in the case of smoke, we can rely on the amount of grey; in the case of tumult, or even explosion, we can use the summation of the intensity of the movement vectors detected in subsequent images/frames; and so on. Just as pointed out in this section, once the content has been retrieved, any kind of visualisation fits on.

2.3.6. Novelty Detection over Data Streams

In what concerns stream analysis, novelty detection has been stated as a multi-class classification problem where the goal is to discriminate examples from the “Normal” classes and the “not-Normal” classes. In the case of RESCUER, we are interested in when, along the time of the stream, the crisis comes to define not-Normal classes. Stream processing algorithms work with vectors of features and with distance functions, so they can handle complex data reasonably straightly following the techniques described in section 2.3.5. More specifically, novelty detection relies on clustering techniques, whereas the clusters are the normal and crises automatically identified according to the features of the data.

Novelty detection depends on a learning phase based only on examples of the normal concept, when crisis is not taking place. Then, in the application phase, the stream of unlabelled examples can be classified as either normal, belonging to the normal concept learned in the training phase, or unknown, not belonging to the normal concept [42]. The unknown examples can indicate the presence of a new class, a novelty, which was not learned in the training phase and that, potentially alerts for an emergent crisis. In novelty detection problems, the normal concept may be composed by different classes, and novel classes may appear in the course of time, resulting in a concept evolution. Thus, the decision model cannot be static, but it should rather evolve to represent the new emergent classes. Since concepts are hardly ever constant, the application of novelty detection in data streams represents an important challenge [43]. For the RESCUER project, we aim at using algorithm MINAS (MultI-class learNing Algorithm for data Streams) to deal with novelty (crisis) detection [44].

It has been intended to use Novelty detection along with visualisation in project RESCUER. While the visualisation uses colour for encoding information – among other visual features, it demands previous intelligence in order to use pre-attentive colouring to call attention for critical events. To this end, Novelty detection will be used to state which arriving data is of greater concern. Once the arriving data is visualised following stream visualisation techniques adapted to a geographical map – location-based, the colour channel may be reserved for indicating what MINAS, running in back-end, has identified. This engineering is supposed to improve the efficacy of the toolkit, as it has the potential of providing a clearer picture of the situation to the rescue experts.

22

3. First Prototype of the Visualisation Mechanisms This chapter presents the different visualisation mechanisms implemented for the prototype of

the first iteration. Section 3.5 summarizes the technical characteristics of these implementations.

3.1. Overview

The first RESCUER prototype of Visualisation Mechanisms focuses on Emergency Mapping. It is a mock-up to show how graphical user interface (GUI) for a web-based emergency mapping application (as the central feature of the Emergency Response Toolkit) might look like. Figure 15 gives an example of the prototype showing different data on a map base layer.

Figure 15: Prototype for visualisation mechanisms for emergency mapping – a map view metaphor

3.2. Layers

The Layer Control (legend) on the right side of Figure 15 consists in five layer groups and each of them has its layers. The layer groups are:

23

• Base Layers offer the style of the map in the background of the information to visualise. It is

possible to show a road map (Figure 15), satellite image (see Figure 16), or a hybrid view with road labels on top of satellite images. The data source for those base layers of the prototype is Microsoft Bing Maps2.

• The Incidents and Event layer groups contain only one layer. While events are aggregated, crowd submissions like reports, pictures and videos, incident information is entered by the dispatcher describing and showing complete affected spatial areas (see the blue polygons in Figure 15).

• Crowd Data basically consists of layers that show information provided by the crowd without any semantics like Events or Incidents. This includes layers for the density of the crowd, shown as heat map (see Figure 16), pictures, messages (reports) and videos. While crowd density is gathered automatically, sending pictures, messages and videos to the command and control centre requires user interaction in the crowd.

• The Safety and Security Areas group currently consists of only one layer to show spatial areas, e.g. where cars are allowed to park.

Figure 16: Prototype of visualisation mechanisms for emergency mapping - satellite view with

weather information

2 http://www.microsoft.com/maps/ 24

3.3. Detail information

For every information item in the map, it is possible to show detailed information in a floating window. The example in Figure 17 shows an event with multimedia attachments. All events are annotated with keywords from categories like who, what and where and have a timestamp. In this case, the event consists of messages (as text), images and video data, sent by different people in the crowd.

Figure 17: Prototype for visualisation mechanisms for emergency mapping - event details and crowd statistics

3.4. Additional Information

Furthermore, it is possible to show additional information like the general weather conditions (see Figure 16), which is of high interest depending of the type of the incident. Especially rain and wind conditions have a strong influence on fire or gas leakages, but also crowd behaviour and bottom conditions, evacuation roads and so on. A crowd statistics window (example shown in Figure 17) displays graphical charts of different factors. In the current mock-up, the prototype gives stacked

25

bar charts for the incident type keywords of all crowd submissions organised by media attachment types.

3.5. Technical Background and Requirements

From a technical point of view, the Emergency Response Toolkit is implemented as an HTML5 interactive Web app using OpenLayers 33 for interactive emergency maps, Highcharts4 for interactive statistics and Twitter Bootstrap5, jQuery6 and jQuery UI7 for general formatting interaction features.

In the first iteration of the prototype, the data is stored in files. It is provided using JSON and KML (Keyhole Markup Language) files as well as media files like images and videos. The first deliverable using real data will be the first iteration of the Emergency Response Toolkit in D4.4.1.

A Web server is required to run the prototype, rather than just opening the HTML file in a browser. This is due to security mechanisms in modern browsers that deny access of local files (such as all data sources in the prototype) via JavaScript. For the first prototype, Google Chrome was the only tested browser.

3 OpenLayers 3: http://ol3js.org/. 4 Highcharts - Interactive JavaScript charts for your webpage: http://www.highcharts.com/. 5 Boostrap: http://getbootstrap.com/. 6 jQuery: http://jquery.com/. 7 jQuery UI: http://jqueryui.com/.

26

4. Second Prototype of the Visualisation Mechanisms This chapter presents the different visualisation mechanisms implemented for the prototype in

the second iteration. Section 4.7 summarizes the technical characteristics of these implementations.

4.1. Overview

The second RESCUER prototype of the visualisation mechanisms focuses on the coordination among workforces. Additionally, the first prototype, which focused on emergency mapping, has been improved. The current version of the Emergency Response Toolkit is already able to gather data from the Mobile Crowdsourcing Solution, therefore this prototype is not a mock up anymore. An overview of the prototype is shown in Figure 18 and explained in the next sections.

Figure 18: Prototype for the visualisation mechanisms for workforces implemented as part of the Emergency Response Toolkit

4.2. Improvements for the Emergency Mapping

Map menu

Since the amount of features available on the Map View has increased, we created a menu to manage them. This menu is draggable to avoid disturbing the user. Figure 19 shows the menu with the cells numbered in order to reference them in the following sections. The menu of the current prototype offers the option to show or hide the features: the controls of the map layers (1), the panel to move the map to a certain location (2), the tables to navigate through incidents (3), reports (4) and workforces (5), and the features to draw a point (6), a line (7) or a polygon (8) in the map. The new features are explained in the following sections.

27

Figure 19: Menu on the map view

Markers Distance Clustering

The Emergency Response Toolkit displays some georeferenced information in the map view, e.g. the incidents. It is very likely that, in an emergency situation, there will be a large number of incidents and most of them close to each other, in terms of position. Placing large number of markers on a map would lead to a bad user experience, since they will overlap. Even if the overlapping is not the problem, they can cause visual overload. This problem gets worse when the map is zoomed out. A distance clustering is a method that simplifies the visualisation by aggregating information of a nearby area.

Figures 20, 21 and 22 show how the clustering is performed. In this prototype, the markers closer than 30 pixels are aggregated and the size of the bubble depends slightly on the amount of clustered elements.

Figure 20: Map view with low zoom

28

Figure 21: Map view with medium zoom

Figure 22: Map view with high zoom

The markers show its information when they are clicked, e.g. an incident marker shows a panel with its basic information. The clustered markers, when clicked, show which elements are encapsulated in it. Figure 23 shows a cluster of incidents after clicking on it. Each of these circles with an icon inside represents an incident and the icon indicates the type of the incident. In this case, there are four incidents clustered, three with the same type. Each incident, when clicked on, does the same as an individual marker, and shows its basic information in a panel. This allows the user to have the same functionality, independently if they are clustered or not.

29

Figure 23: Incidents clustered in Map View

QR Codes in Map Layer

The ERTK prototype runs in different devices at the same time. This proposal reaches a good solution to enable users to access the prototype from a different device effortlessly. A QR Code is a link visualisation, which supports the use case of opening an incident detail page from another device without typing any link; one just needs to scan the QR code from the map view. Figure 24 illustrates how to visualise a QR code of an incident.

Figure 24: Incident information with a QR code as a link for the details page

30

Map Annotations

The map view offers the visualisation of the incident area with the incidents and workforces. However, it is also useful to let the user create and visualise his own notes over the map. The proposal is to have a drawing feature with the possibility to annotate points, lines and polygons for the sake of indicating observations of interest. Figure 25 shows a map view with annotations using the drawing features. Different points can be seen as interesting points. The annotations with lines may indicate routes while annotations with polygons can depict interesting areas.

To work with the drawing feature, the user should go to the map view and choose between the three types of available drawings: point, line and polygon. Each option is shown with a palette icon and the type of painting. After clicking on one of those options, the drawing functionality is enabled. If the option is clicked again, the functionality is disabled. The drawing can be performed as follows:

• Point: The feature of drawing a point can be made just by clicking on the map where we

want to add this point, and it is automatically added. • Line: the line can be composed by several points. Therefore, the user should click on the

interested points. In the last point, the user should make a double click to indicate the end of the line.

• Polygon: A polygon is created as a line, except for the end. The user should click again on the first point to indicate the end of the polygon.

The current propotype has only implemented the drawing feature itself. It is not yet implemented neither the storage of these drawings nor the feature to remove or modify a certain drawing. These implementations are part of Task 4.4.

31

Figure 25: Map View with drawing annotations added

Incident Navigation in Map View

The purpose of this feature is to allow the user to find an incident on the map without previously knowing its position. Therefore, a list of incidents with their main information is offered. The user can navigate through it and click on the desired incident. The selected incident is highlighted on the map. Figure 26 shows a table of incidents with one row selected, which indicates the incident chosen by the user. The incidents are visualised with blue markers on the map. In the figure, there is a yellow marker, which is the marker of the selected incident. The user can easily identify where this incident takes place.

In order to get this table displayed on the map, the user should click on the cell of the map menu numbered as 5 in Figure 19. The same functionality exists with reports and workforces. However, when a report is selected, the incident containing this report is highlighted.

32

Figure 26: Map View with an incident marker selected from a table

4.3. Workforces Map Layer in Map View

The position of the workforces is an important information for the coordination. Therefore, they have been added in the map view. Figure 27 shows the workforces on the map and one of them selected. If the user clicks on a workforce, this one is highlighted and a panel with his/her main information is displayed.

Figure 28 shows a workforce in the map highlighted, selected from the table. This shows the same functionality “Incident Navigation in Map View” explained in Section 4.2.

Figure 27: Map view with the layer of workforces

33

Figure 28: Map view with a workforce highlighted in the table

4.4. Incident Detail

The information of an incident is visualised in the “Incident Detail View” (Figure 29). It has four main parts: a table with basic information, a timeline, a pie chart and a map. The table presents the following basic information:

• Timestmap: when this incident has started; • Last Updated Timestamp: last time when it has been updated; • Keyword: type of incident; • Reliability: how trustable is this incident depending on certain factors; e.g. the role of the

people who has reported this incident; • Reports: Number of reports that contains this incident; • Merged Incidents: show if there are other incidents which refer to the same occurrence.

The timeline places the reports of a given incident on the timeline in accordance to the moments when they were sent, providing an overview of how long this incident is taking place or in which moment it had more activity. The pie chart illustrates the roles of the people who sent reports related to a given incident (e.g., Visitor, Operational Force Member, Employee). The map shows the position of the reports sent regarding this incident.

34

Figure 29: View of the incident details

4.5. Workforces and Tasks Coordination

In an emergency situation, the workforces working to cope with the incidents and the tasks assigned to these workforces are important information for the coordination. Hence, a view only for these has been added and it has been called “Coordination View”. Two tables are used to show the tasks and the workforces, one for each. The user of the prototype can create tasks and assign workforces to it. When a task is created, a geographical point can be selected, defining where this task should take place. The map at the bottom shows the position of the tasks and the workforces. If a task or a workforce is selected, either on the map or in the table, it is highlighted. The user can visualise clearly the relationship between the elements.

Figure 30 shows an example of this view, with a task selected and its three workforces assigned. Besides, a line is visualised on the map between the task and the assigned workforce. This line indicates the distance in meters or kilometres between them. This allows the user to have a better perspective of the distribution of incidents and workforces.

35

Figure 30: Coordination View

4.6. Emergency Overview

The information about the emergency situation is visualized in the ERTK prototype through several views (e.g., map view, incidents browser view, and others). Each view shows in detail one part of the overall situation. Therefore, a section to get an overview of the whole information is needed. For this purpose, an emergency overview has been added, as shown in Figure 31. This view provides the main information regarding the general situation:

• Status of the emergency; • Position and time of the emergency situation • Timeline of the reports received • Timeline of the incidents • Tasks and workforces classified by their status; and • Types of the incidents with their frequency of occurrence.

The timeline graphs of the reports and incidents show the times when they were sent or created, giving an overview of how long the emergency is taking place or the different levels of activity depending on the time. In a real case, it is expected to have a lot of incidents created at the beginning of the emergency, with an increasing trend, and a decreasing curve during the last intervals. The tasks and workforces diagrams are meant to inform with a quick look to the user if the workforces have already arrived at the place, how many members of the workforce are available, and if the assigned tasks are finished or still on progress. The different types of incidents occurred,

36

(called keywords) are shown, each of them with a different text size to indicate which has been more often identified.

Figure 31: Emergency Overview

4.7. Technical Background and Requirements

The technical background and requirements for the first iteration, detailed in Section 3.5, are the same for the second iteraction except that the data is not stored in files but real data is received. The data is still formated using JSON.

37

5. Conclusions The preliminary literature review depicts that current emergency systems make use of maps and

icon-based paradigm. More relevant, it seems to be the fact that in most situations related to emergency crisis management, both sides are involved: the population and support personnel. The underlying need for cooperation between those parties evolve the requirement for a collaborative environment, as well as mechanisms for custom visualisation to aid in this context. The choice of suitable devices for these custom visualisations lead to the need of adapting the viewing environment for each of those devices.

For stream visualisation in Section 2.3, the background mechanisms for video analysis processing have been investigated, in order to identify proper visualisation concepts for their results. In comparison, existing visualisation approaches for streams have been investigated. The observation depicted that stream visualisations especially make use of metaphors and analogies to the real physical behaviour of objects. Subsection 2.3.5 described features, distance functions, and metric spaces and, with respect to the results of sections 2.1 and 2.2 an appropriate stream visualisation for RESCUER has been discussed, taking the novelty detection algorithm MINAS over data streams into account, which will be applied in RESCUER.

The possible visualisation mechanisms for the information regarding the coordination of the workforces and tasks were investigated and the results are described in section 2.4.

As stated in Section 2.2, the main goal of existing emergency response toolkits in the literature is to provide knowledge through views that support decision-making quickly and assertively. In this literature review, especially, a gap was observed to compare and evaluate these works to align the state of the practice in order to advance the state of the art. This should be investigated in the next iteration of this task, because we believe that the RESCUER project will go on this direction, since it plans to focus on the trends of the area (e.g. use of maps, layers, mechanisms of interaction), and it will also be evaluated on real scenarios.

The visualisation mechanisms implemented in the first iteration have been analysed in terms of the RESCUER requirements and they were improved. The results of the evaluations of the ERTK performed with end-users, and described in the Deliverable D5.3.1 [58], have been applied using more appropriate metaphors and interaction concepts. The main improvements were described in Section 4.2 and they were applied in the “Map View”. These improvements were implemented in the scope of the Task 4.4, which is the implementation of the Emergency Response Toolkit. During this task, new visualisation techniques related to the first iteration came up. Besides, after implementing more parts of the ERTK, more concepts which need a better visualisation arise.

38

References [1] Charu C. Aggarwal, Alexander Hinneburg, and Daniel A. Keim. On the surprising behavior of

distance metrics in high dimensional spaces. In Jan Van den Bussche and Victor Vianu, editors, International Conference on Database Theory (ICDT), pages 420–434, London, UK, 2001. Springer Verlag.

[2] Wolfgang Aigner, Silvia Miksch, Wolfgang Müller, Heidrun Schumann, and Christian Tominski. Visualizing time-oriented data-a systematic view. Comput. Graph., 31(3):401–409, June 2007.

[3] J. Alsakran, Yang Chen, Dongning Luo, Ye Zhao, Jing Yang, Wenwen Dou, and Shixia Liu. Real-time visualization of streaming text with a force-based dynamic system. Computer Graphics and Applications, IEEE, 32(1):34–45, Jan 2012

[4] Brian Babcock, Shivnath Babu, Mayur Datar, Rajeev Motwani, and Jennifer Widom. Models and issues in data stream systems. In Proceedings of the Twenty-first ACM SIGMOD-SIGACT-SIGART Symposium on Principles of Database Systems, PODS ’02, pages 1–16, New York, NY, USA, 2002. ACM.

[5] Bertin, Jacques: Semiology of graphics. University of Wisconsin Press, 1983 [6] Eric A Bier, Maureen C Stone, Ken Pier, William Buxton, and Tony D DeRose. Toolglass and Magic

Lenses: The See-through Interface. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’93, pages 73–80, New York, NY, USA, 1993. ACM.

[7] Nan Cao, Yu-Ru Lin, Xiaohua Sun, D Lazer, Shixia Liu, and Huamin Qu. Whisper: Tracing the Spatiotemporal Process of Information Diffusion in Real Time. Visualization and Computer Graphics, IEEE Transactions on, 18(12):2649–2658, 2012.

[8] B. Chandramouli, M. Ali, J. Goldstein, B. Sezgin, and B.S. Raman. Data stream management systems for computational finance. Computer, 43(12):45–52, Dec 2010.

[9] J A Cottam, A Lumsdaine, and C Weaver. Watch this: A taxonomy for dynamic data visualization. In Visual Analytics Science and Technology (VAST), 2012 IEEE Conference on, pages 193–202, 2012.

[10] N Elmqvist, P Dragicevic, and J Fekete. Color Lens: Adaptive Color Scale Optimization for Visual Exploration. Visualization and Computer Graphics, IEEE Transactions on, 17(6):795–807, 2011.

[11] Jean-Daniel Fekete and Catherine Plaisant. Excentric Labeling: Dynamic Neighborhood Labeling for Data Visualization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’99, pages 512–519, New York, NY, USA, 1999. ACM.

[12] Joaquim Cezar Felipe, Agma Juci Machado Traina, and Caetano Traina Jr. Global warp metric distance: Boosting content-based image retrieval through histograms. In Jeffrey J. P. Tsai, Max Muhlhauser, and Makoto Takizawa, editors, IEEE International Symposium on Multimedia-ISM2005, page 8, Irvine, CA, 2005. IEEE Press.

[13] B.J.Fry. OrganicinformationDesign. PhDthesis, MassachusettsInstituteofTechnology, MA, USA, 2000.

[14] G W Furnas. Generalized Fisheye Views. SIGCHI Bull., 17(4):16–23, 1986. [15] Mohamed Medhat Gaber, Arkady Zaslavsky, and Shonali Krishnaswamy. Mining Data Streams: A

Review. SIGMOD Rec., 34(2):18–26, 2005. [16] Harmelen, Frank van, Jeen Broekstra, Christiaan Fluit, Herko ter Horst, Arjohn Kampman, Jos van

der Meer and Marta Sabou: Ontology-Based Information Visualisation. 2001.

39

[17] Herman, I., Guy Melancon und M.S. Marshall: Graph visualization and navigation in information

visualization: A survey. Visualization and Computer Graphics, IEEE Transactions on, 6(1):24 - 43, Jan-Mar 2000

[18] Samuel Huron, Romain Vuillemot, and Jean-Daniel Fekete. Bubble-TV: Live Visual Feedback for Social TV Broadcast. In CHI 2013 Workshop : Exploring and enhancing the user experience for television, Paris, France, 2013.

[19] Samuel Huron, Romain Vuillemot, and Jean-Daniel Fekete. Visual sedimentation. IEEE Transactions on Visualization and Computer Graphics, 19(12):2446–2455, December 2013.

[20] G. L. Byungkyu Kang and T. H. llerer. Tweetprobe: A real-time microblog stream visualization. In Proc. IEEE VIS Arts Program (VISAP), pages 1–7, 2013.

[21] Keim, Daniel A.: Databases and Visualization. In: Jagadish, H. V. und Inderpal Singh Mumick (Herausgeber): SIGMOD Conference, Seite 543. ACM Press, 1996

[22] M Krstajic, E Bertini, and D A Keim. CloudLines: Compact Display of Event Episodes in Multiple Time-Series. Visualization and Computer Graphics, IEEE Transactions on, 17(12):2432–2439, 2011.

[23] Y Lin and R Vuillemot. Spirograph designs for ambient display of tweets. In Proc. IEEE VIS Arts Program (VISAP), pages 8–15, 2013.

[24] Mackinlay, Jock: Automating the Design of Graphical Presentations of Relational Information. ACM Transactions on Graphics, 5:110 - 141, 1986

[25] Florian Mansmann, Milos Krstajic, Fabian Fischer, and Enrico Bertini. Streamsqueeze: a dynamic stream visualization for monitoring of event data. In Proc. SPIE, volume 8294, pages 829404–829404–12, 2012.

[26] A Norton, M Rubin, and L Wilkinson. Streaming graphics. Statistical Computing and Graphics Newsletter, 12(1):11–14, 2001.

[27] Ken Perlin and David Fox. Pad: An Alternative Approach to the Computer Interface. In Proceedings of the 20th Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’93, pages 57–64, New York, NY, USA, 1993. ACM.

[28] Maria Popova. Data visualization: Stories for the information age. BusinessWeek, agosto, 2009. http://www.businessweek.com/innovate/content/ aug2009/id20090811\_137179.htm, fevereiro, 2013.

[29] Qeli, Ermir: Information Visualization Techniques for Metabolic Engineering. PhD, Philipps-Universität Marburg, Fachbereich Mathematik und Informatik, Prof. Dr. Bernd Freisleben, 2007.

[30] Shneiderman, Ben: The future of interactive systems and the emergence of direct manipulation. In: Proc. of the NYU symposium on user interfaces on Human factors and interactive computer systems, Seiten 1{28, Norwood, NJ, USA, 1984. Ablex Publishing Corp

[31] Shneiderman, Ben: Human-computer interaction: a multidisciplinary approach, Kapitel Direct manipulation: A step beyond programming languages, Seiten 461 - 467. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1987

[32] H. Thakkar, N. Laptev, H. Mousavi, B. Mozafari, V. Russo, and C. Zaniolo. Smm: A data stream management system for knowledge discovery. In Data Engineering (ICDE), 2011 IEEE 27th International Conference on, pages 757–768, April 2011.

[33] James J. Thomas and Kristin A. Cook, editors. Illuminating the Path: The Research and Development Agenda for Visual Analytics. IEEE Press, 2005. 184 pp. online: http://nvac.pnl.gov/agenda.stm.

40

[34] Fernanda Viégas, Ethan Perry, Judith Donath, and Ethan Howe. Artifacts of the presence era:

Visualizing presence for posterity. In ACM SIGGRAPH 2004 Sketches, SIGGRAPH ’04, pages 19–, New York, NY, USA, 2004. ACM.

[35] Voss, Viktor: Aufgabenorientierte Datenvisualisierung für die semantikunterstützte Informationssuche im Finanzsektor. Diplomarbeit (Master Thesis), Technische Universität Darmstadt, Fraunhofer Institut für Graphische Datenverarbeitung, IGD, 2008.

[36] R Vuillemot, S Huron, R Velt, and Y M Houssonne. Bubble–T: Et si twitter votait? http://dev.fabelier.org/bubble-t/.

[37] Wijk, Jarke J. van: Views on Visualization. IEEE Trans. Vis. Computer Graphics, 12(4): 421-433, 2006.

[38] Pak Chung Wong, H Foote, D Adams, W Cowley, and J Thomas. Dynamic visualization of transient data streams. In Information Visualization, 2003. INFOVIS 2003. IEEE Symposium on, pages 97–104, 2003.

[39] Zaixian Xie, MatthewO. Ward, and ElkeA. Rundensteiner. Visual exploration of stream pattern changes using a data-driven framework. In Advances in Visual Computing, volume 6454 of Lecture Notes in Computer Science, pages 522–532. Springer Berlin Heidelberg, 2010.

[40] D. S. Zhang and G. J. Lu. Shape retrieval using fourier descriptors. In Intl. Conference on Multimedia and Distance Education, pages 1–9, Fargo, ND, USA, 2001.

[41] C Ziemkiewicz and R Kosara. The Shaping of Information by Visual Metaphors. Visualization and Computer Graphics, IEEE Transactions on, 14(6):1269–1276, November 2008.

[42] E. J. Spinosa, A. C. P. L. F. Carvalho, and J. Gama. Novelty detection with application to data streams. Intelligent Data Analysis, 13(3):405–422, 2009.

[43] J. Gama. Knowledge Discovery from Data Streams. Chapman & Hall/CRC Data Mining and Knowledge Discovery Series. CRC Press, 1 edition, 2010.

[44] Elaine R. Faria, João Gama, André C. P. L. F. Carvalho: Novelty detection algorithm for data streams multi-class problems. SAC 2013: 795-800.

[45] Manlai You, Chun-wen Chen, Hantsai Liu, and Hsuan Lin : A Usability Evaluation of Web Map Zoom and Pan Functions, 2006

[46] Ji Soo Yi, Youn ah Kang, John T. Stasko: Toward a Deeper Understanding of the Role of Interaction in Information Visualization, 2007

[47] Anna Wu, Gregorio Convertino, Craig Ganoe, John M. Carroll, Xiaolong (Luke) Zhang, Supporting collaborative sense-making in emergency management through geo-visualization, International Journal of Human-Computer Studies, Volume 71, Issue 1, January 2013, Pages 4-23.

[48] Mohamed Medhat Gaber, Shonali Krishnaswamy, Brett Gillick, Hasnain AlTaiar, Nicholas Nicoloudis, Jonathan Liono, Arkady Zaslavsky, Interactive self-adaptive clutter-aware visualisation for mobile data mining, Journal of Computer and System Sciences, Volume 79, Issue 3, May 2013, Pages 369-382.

[49] Junghoon Chae, Dennis Thom, Yun Jang, SungYe Kim, Thomas Ertl, David S. Ebert, Public behavior response analysis in disaster events utilizing visual analytics of microblog data, Computers & Graphics, Volume 38, February 2014, Pages 51-60.

[50] Rico Richter, Jürgen Döllner, Concepts and techniques for integration, analysis and visualization of massive 3D point clouds, Computers, Environment and Urban Systems, Volume 45, May 2014, Pages 114-124.

41

[51] David Díez, Sara Tena, Rosa Romero-Gomez, Paloma Díaz, Ignacio Aedo, Sharing your view: A

distributed user interface approach for reviewing emergency plans, International Journal of Human-Computer Studies, Volume 72, Issue 1, January 2014, Pages 126-139.

[52] Engelbrecht, A.; Borges, M.; Vivacqua, A.S., Digital tabletops for situational awareness in emergency situations, Computer Supported Cooperative Work in Design (CSCWD), 2011 15th International Conference on , vol., no., 8-10 June 2011, Pages 669-676.

[53] Yong Liu, Pratch Piyawongwisal, Sahil Handa, Liang Yu, Yan Xu, Arjmand Samuel, Going Beyond Citizen Data Collection with Mapster: A Mobile+Cloud Real-Time Citizen Science Experiment, IEEE International Conference on e-Science Workshops, eScienceW 2011, 5-8 December 2011, Pages 1-6.

[54] Khandaker Mustakimur Rahman, Tauhidul Alam, Mahfuzulhoq Chowdhury, Location Based Early Disaster Warning and Evacuation System on Mobile Phones Using OpenStreetMap, IEEE Conference on Open Systems (ICOS), 21-24 October 2012, Pages 1-6.

[55] Jobst Löffler, Vera Hernández Ernst, Jochen Schon, Jens Pottebaum, Rainer Koch Intelligent Use of Geospatial Information for Emergency Operation Management, Intelligent Human Computer Systems for Crisis Response and Management, 13-16 May 2007.

[56] Tomoyuki Ishida , Akira Sakuraba, Noriki Uchida, Koji Hashimoto, Yoshitaka Shibata. A Unified Large Scale Disaster Information Presentation System Using Ultra GIS based Tiled Display Environment, International Conference on Network-Based Information Systems, 26-28 September 2012, Pages 550 – 555.

[57] Ben Shneiderman, The Eyes Have It: A Task by Data Type Taxonomy for Information Visualizations. In Proceedings of the IEEE Symposium on Visual Languages, pages 336-343, Washington. IEEE Computer Society Press, 1996.

[58] RESCUER. Deliverable D5.3.1. Evaluation Report of the Emergency Response Toolkit 1.

42