White Paper

Hexagon’s Luciad Technology and

Visual Analytics (VA)

18 April 2019

2

Contents Challenges for Geospatial Visual Analytics ........................................................................ 3

Building Real-time Visual Analytics Applications .............................................................. 3

Data Fusion ......................................................................................................................... 3

Data Abstraction .................................................................................................................. 5

Multiple Representations ..................................................................................................... 7

Interaction ............................................................................................................................ 9

Scalability .......................................................................................................................... 11

Visual Analytics Application Examples ............................................................................. 11

Exploring Geospatial Big Data – 20 Years of History in Africa ........................................... 11

Exploring Visual Analytics in an Amusement Park Setting ................................................. 12

Rerouting Flights in Real Time ........................................................................................... 13

Researching the Spread of Diseases Over Time ............................................................... 13

Analyzing Big Telecom Data .............................................................................................. 14

Analyzing Data from a Ski Trip in the Alps ......................................................................... 15

Assessing Risk Involving Oil Rig Permits and Maintenance .............................................. 15

Visually Analyzing Millions of Twitter Feeds ...................................................................... 16

Contact Us ........................................................................................................................... 17

About Hexagon.................................................................................................................... 17

3

Over the last two decades, data has been produced at an incredible rate. However, the ability to collect and store this data is increasing at an even faster rate - faster than the ability to analyze it.

Visual Analytics (VA) is the science of analytical reasoning facilitated by interactive visual interfaces (Thomas and Cook1). VA techniques facilitate the analysis of real-time data streams by presenting the results in a meaningful and intuitive way while allowing interaction with the data. These techniques enable quick identification of important information and timely reaction to critical process states or alarming incidents.

Challenges for Geospatial Visual Analytics VA plays an important role in many domains, including those dealing with spatio-temporal data. For implementing VA in geospatial software that uses Luciad technology, five key challenges are promoted:

• Data fusion – bringing together different data sources

• Data abstraction – visualizing your data in an abstract way to highlight or bring forward what is important

• Representing your data in multiple ways – VA is about choosing the best visual representation, or even multiple representations

• Interaction is required for a dynamic VA application

• The visualization itself should be scalable. It must remain fluent, smooth, and intuitive, even when working with large data sets.

Below, the above challenges are elaborated upon. Examples are used to illustrate how Luciad technology is different and meets all five key challenges, making it the ideal spatio-temporal engine for any truly interactive VA application.

Building Real-time Visual Analytics Applications

Data Fusion

In most VA applications, heterogeneous data sources need to be integrated before any VA can be applied. Therefore, the first step is often to preprocess and transform the data in order to extract meaningful units of data for further processing.

As an alternative to preprocessing data, Luciad technology provides a wide variety of data connectors that allow applications to ingest every type of data. The ability to directly connect to any data source is fundamental, even if the data source is a format that is not optimized for the platform. This means you can directly ingest OGC WFS with GML in a web environment, KML data, GeoTIFF files, shapefiles, and so on without having to pre-process and convert to a geodatabase. In addition, the Luciad platform provides extension interfaces and a Unified Domain Model to implement a connector for any custom or proprietary format, ensuring you immediately benefit from the visualization capabilities.

A requirement for geospatial VA is the ability to handle any coordinate reference system and projection, and then to transform vector and raster data on the fly. This includes vector transformations, discretization, raster warping, geodetic and rhumb line discretization, and correct handling of projection boundaries. In many domains, using the correct geographical projection is critical. For example, to accurately analyze vessel traffic around the North Pole, you must be able to use a polar stereographic projection for all your data sources. The Luciad platform does so, even in its web SDK, LuciadRIA, which can visualize any data

1 James J. Thomas and Kristin A. Cook (Ed.) (2005). Illuminating the Path: The R&D Agenda for Visual Analytics National Visualization and Analytics Center.

4

source in any 2D or 3D projection. Professional applications are concerned about preserving angle and area. Good visual interpretation requires the use of the correct and most appropriate reference.

Figure 1: Luciad technology handles any coordinate reference system and projection.

An engine for geospatial VA also needs to handle advanced shapes, far beyond simple points, Cartesian lines, and Cartesian polygons. In many domains, including Defense and Aviation, geodesic lines, rhumb lines, arcs, circles, ellipses, arc bands, buffers, and so on are crucial. These shapes need to be automatically discretized and visualized in any projection, without requiring the developer to introduce intermediate points or to implement discretization code. Luciad technology handles all these aspects automatically.

When dealing with geospatial data in a 3D world, an important limitation that most classical GIS engines suffer from is the inability to easily represent data in both 2D and 3D. This mostly applies to data such as streets and annotations on the terrain. In 2D, painting different layers is easy and straightforward – using the painter’s algorithm, you simply render the layers back to front. In 3D, this does not hold: for layers that should be draped on the terrain, different strategies are adopted by geospatial software providers, all suffering from important drawbacks that render the engines incapable of performing interactive visualization. The two most common adopted strategies for draping are:

• Rasterize data to tiles, which are then applied to the terrain. This serious drawback means that all of your data must be tiled to begin with. This fails to meet the requirement to be able to handle any data source, such as video and radar feeds.

• Use stencil buffer techniques to determine which terrain pixels should be covered by the draped shapes. The serious drawback of this technique is that it can only be used for vector shapes. It also precludes batching of multiple shapes, which is important to maintain performance when using hardware-accelerated rendering. Video draping, radar draping, or draping of heat maps is not possible.

Luciad technology overcomes these limitations by using a proprietary algorithm that allows draping of any data source - vector or raster data, tiled or not, static or dynamic, video or radar feeds, and so on. The algorithm even works for custom layer and painter implementations. All of this is accomplished with no performance difference compared to visualizing data in 2D. In fact, no distinction is made between a 2D or a 3D layer in the API, or between a draped shape or a non-draped shape. Luciad Portfolio products offer a single API for 2D and 3D and allow visualization of any shape, layer, or data source draped on the terrain.

5

Figure 2: Video draping from UAV feed.

Data Abstraction

An essential element of VA is to create a high-level view of the dataset, while still maximizing the amount of detail. Representing important features while hiding away the implementation details is referred to as data abstraction.

Let’s illustrate this with a simple example, but one which is difficult to achieve using traditional GIS APIs. Assume you have a GeoJSON file of countries. Each country has different properties, including its name, shape, and demography statistics. Most geospatial APIs would be able to visualize each country’s shape. But what if you also want to visually display the demography numbers by category using a pie chart? With Luciad technology, this is extremely simple to achieve, as Luciad Portfolio products cleanly separate the countries and their properties from how they are visualized. This is known as model-view separation.

Figure 3: A pie chart displayed on top of a country.

Another powerful data abstraction technique is aggregation. Instead of visualizing every individual feature, a summary, or aggregation, is displayed. This is what enables VA to present an overview to the user first, and then let him drill down to discover the details.

Two powerful VA methods to perform data aggregation are clustering and density plots (or heat maps). Luciad technology offers both data aggregation techniques with no compromises: both work for 2D or 3D, draped or not, and for static or dynamic (4D) data.

By determining which objects in a visible area are similar, and by displaying similar objects as one clustered object, your map quickly clears up. As you zoom into the map, the object cluster dissolves again into distinct objects.

6

Luciad technology offers highly configurable clustering options for your data sets. You can cluster objects such as airport NOTAM notifications, military icons and mission reports, or any other point-related objects. The platform allows you to set up clustering algorithms that aggregate point-object icons based on a multitude of criteria. You can simply cluster objects based on the distance between them, and just as easily base your clustering logic on object properties available from the point metadata, such as class or hierarchy. For example, in a military symbology data set, you can decide to cluster symbols based on the order of battle, or to cluster only certain types of symbols, such as land units. Other symbol types can optionally be let unclustered at all times.

The visualization of your cluster is also fully customizable and configurable: you have control over cluster size, cluster styling, clustering zoom levels, and even cluster positioning. In a set of military symbols, you can make sure that friendly forces clusters are not accidentally positioned in enemy territory, for instance.

Since clustering can be highly domain- or use-case specific, the Luciad Portfolio offers a powerful and flexible API to avoid clustering objects from different groups together – for example, enemy versus friendly forces – or to avoid clustering objects across country borders, such as avoiding placing your troops in unsafe territory.

Figure 4: Similar objects displayed as one clustered object. When zooming in, the object cluster dissolves again into distinct objects.

Heat mapping, from a geographic perspective, is a method of showing the geographic assembling of a phenomenon. The resulting density surface is visualized by using a gradient that allows the areas of highest density (or hot spots) to be easily identified. Luciad APIs can compute density plots or heat maps on the fly for any shape (points, lines, polygons, or other shapes) and combine them with interactive filtering.

7

Figure 5: Density plot of ski tracks.

Multiple Representations

Using multiple representations is another powerful VA technique. In the LuciadRIA example in Figure 6, you see the same demography statistics that were shown in the pie chart visualization in Figure 3. To get a global overview, it’s more interesting to visualize each statistic on a separate map, with yellow representing high percentage concentration, and red representing low percentage concentration.

Figure 6: A heat map representation of demography statistics.

For example, in the upper left map you immediately see that African countries have the highest percentage of female population younger than 15. On the maps in the right column, on the other hand, you see that Africa has the lowest percentage of elderly people. Thanks to model-view separation, Luciad Portfolio products allow you to visualize the same objects on multiple maps without data duplication, while still using the same styling and visualization on all different representations.

In this visualization of a sailing race, you see a map on the right, with various non-geospatial parameters like wind, speed, and optimal direction displayed on the left.

8

Figure 7: Multiple representations of a sailing race.

Figure 8 shows another example of representing the same data in multiple ways.

Figure 8: Multiple representation of flight trajectory in air space.

9

From the top-left 2D view, it is unclear whether the green flight trajectory passes through restricted military airspace (indicated in red). From the top-right 3D view, you get a better impression. But the most accurate visualization is the bottom view, showing the flight profile and the relevant airspace crossings. From this visual representation, it is immediately clear that the flight trajectory does not conflict with the military airspace.

The ability to visualize the same data in multiple ways – including 2D, 3D, or non-georeferenced views - is something that most GIS engines lack. Resorting to multiple different products - for example, to provide 2D and 3D viewing in a web environment - loses the ability to use the same styling in all visual views. This leads to (sometimes slightly) different visualizations, further contributing to confusion and prohibiting visual correlation. Luciad technology offers a single API for not only 2D and 3D visualization, but also for non-geographical visualization such as vertical profile views, scatter plots, and timeline views.

Another way to detect variation or correlation between images is to perform side-by-side comparisons in quick succession. With the swipe controller, users can swipe between two layer sets that cover the same area. With the flicker controller, users can toggle the visibility of layer sets, swapping one layer set out for another with a single mouse click. The porthole controller allows users to peek through and compare layers. These controllers work with any data set, be it vector or imagery data sets, and in 2D or 3D.

Figure 9: Visually comparing two multispectral SWIR images with the swipe controller.

Interaction

User interaction with the visualization is required to reveal insightful information, for instance by zooming in on different data areas or by applying different styling to the data. Filtering allows the users to focus on what’s important to them, and styling can be used to draw attention to the most relevant features.

Many classical GIS APIs store style information with the domain objects and encode it. Some instances, such as encoding as vertex attributes for GPU rendering, prevent data abstraction by violating the model-

10

view separation paradigm. Changing styling or styling expressions requires iterating over all objects. This iteration does not scale well to larger data sets.

Instead, Luciad Portfolio products make object properties, such as feature name, velocity, fuel burn, and so forth, directly available to the GPU and allow you to map properties to style directly on the GPU. This means that expressions are used for styling and filtering, based on the inherent properties of objects, which typically do not change. Changing an expression is extremely fast - often instant - because it passes the expression or expression parameters to the GPU. Since the GPU evaluates these expressions in parallel on hundreds or even thousands of cores, changing expressions is instant.

Let’s examine an example with recorded flight trajectories. We can color those trajectories based on class: Narrowbody Jet, Widebody Jet, and Turboprop.

Figure 10: Flight trajectories colored based on class.

The painter that visualizes these trajectories with a distinctive color looks like this:

The GPU applies the correct color to each trajectory based on the Class property.

Figure 11: Flight trajectories highlighting Widebody Jet in red.

Changing the style to, for example, highlight the Widebody Jet objects in red is instant by invoking the following code:

Note that iterating over the individual trajectories is not required.

var linePainter = new ParameterizedLinePainter({

properties: ["FlightNumber", "Type", "Class"],

propertyColorExpressions: [

{property: "Class", value: "Narrowbody Jet", color: "#FF603E"},

{property: "Class", value: "Widebody Jet", color: "#CAFF41"},

{property: "Class", value: "Turboprop", color: "#8A98FF"}]

});

linePainter.propertyColorExpressions = [

{property: "Class", value: "Narrowbody Jet", color: "white"},

{property: "Class", value: "Widebody Jet", color: "red" },

{property: "Class", value: "Turboprop", color: "white"}]

11

The same techniques are applied for more advanced filtering or styling. For example, you could filter a trajectory based on a property along a trajectory, such as fuel burn, or replay a trajectory by only visualizing the relevant segment. This is used in the award-winning LuciadRIA 3D demo web application to visualize, replay, and analyze 70000 trajectories consisting of a total of 3.5 million data points. Replay, filtering, and interactive styling are instant, leading to a truly interactive VA application.

Scalability

Scalability is a key challenge of VA, as it determines the ability to process large datasets by using computational overhead as well as the appropriate rendering techniques. Users expect a smooth response, even in web applications or when working with large 4D data sets. The goal of Luciad-based applications is 60 FPS, even with large and dynamic data sets. This is achieved by using hardware acceleration (OpenGL, WebGL, OpenCL), intelligent batching, multi-threading, throttling, asynchronous event handling, and the GPU-assisted styling and filter updates discussed above.

For demonstration purposes, Hexagon compared the performance of replaying flight trajectories with the LuciadRIA browser solution versus another geospatial web API. This required 0.5 ms for a single trajectory in the competing product. This might seem like very little time, but 70000 simultaneous trajectories required 35 seconds! This means that if you were to adjust the time slider of this app to examine a different period of time, the application would freeze for more than half a minute while the app updated. This is clearly not acceptable when building true VA applications. Replaying the same trajectories in LuciadRIA, the app updated instantaneously when the time slider was adjusted.

Hexagon has designed and built its Luciad Portfolio products with all the above aspects in mind, and hence offers the only true geospatial technology for 4D VA.

Visual Analytics Application Examples Various domains where Luciad technology was used to build state-of-the-art VA applications are highlighted. For each application, you can view a demonstration movie that showcases the interactivity that can be obtained with Luciad Portfolio products.

Exploring Geospatial Big Data – 20 Years of History in Africa

Hundreds of thousands of diplomatic cables spanning 20 years of history in Africa are visually summarized

simultaneously in three interlinked panels: a map, a word cloud, and a timeline. Focus on a word in the

word cloud and see where and when that organization was mentioned over a 20-year time period.

LuciadLightspeed visualization software harnesses the full power of SAP HANA to provide a geospatial

view on unstructured data. An analysis including Ethnic and Administrative boundaries helps users

understand the events and determine patterns.

12

Link to demonstration movie: Visualizing Unstructured Big Data with Luciad and SAP HANA

Exploring Visual Analytics in an Amusement Park Setting

Follow visitor movements around a fictional theme park Dino Fun World for an entire day in this simulation data set. Analyze group behavior and identify suspicious activity related to a crime in the park.

Link to demonstration movie: Visual Analytics and Group Detection using LuciadLightspeed

13

Rerouting Flights in Real Time

Severe weather can have a significant impact on flight operations. Lufthansa Systems monitors current flight situations in real time using this solution that combines SAP HANA and LuciadLightspeed. Impacted flights can be rerouted in real time, taking into account live weather forecasts and cost-related parameters. Example of volcanic ash cloud impact on flights is shown.

Link to demonstration movie: Air Traffic Management with Luciad Technology

Researching the Spread of Diseases Over Time

Mapping the geographic paths of migrating diseases along with the speed that they are migrating can help researchers better understand the spread of diseases over time.

Using a tool developed for University of Leuven, Department of Microbiology and Immunology, phylogenetic trees of Hepatitis C virus (HCV) are visualized, analyzed, and filtered on a geographic map and time filter. Longer geographic jumps on the map indicate branches that quickly traveled over large distances, while short jumps indicate branches that slowly traveled over relatively small distances.

14

Link to demonstration video: Analyzing Hepatitis C Virus Migration with LuciadRIA

Analyzing Big Telecom Data

From data collected for each cell phone tower in France on (roaming) calls and text messages, see the impact on the network of important events such as extreme weather, in this case, flooding. LuciadLightspeed is used for both the spatial view and the temporal view. Data fusion occurs, for example, by fusing the different CSV files containing cell phone usage information and weather information.

Link to demonstration movie: Extreme Weather Impact on Telecom Network

15

Analyzing Data from a Ski Trip in the Alps

Visual analysis of a week’s worth of GPS data from a ski trip to Mayrhofen ski resort in the Alps. This interactive VA application shows trajectory replay, distinctive styling, density plots, and draping in 3D. The full application integrates terrain data, imagery, KML data, GPX tracks, and more.

Link to demonstration movie: Ski Trip Tracking in the Alps with LuciadRIA

Link to blog post: Ski Trip Throwback with Lucy Test Drive

Assessing Risk Involving Oil Rig Permits and Maintenance

In Oil & Gas, petrochemical, and other high-risk industries, permit-to-work (PTW) systems are used to request, review, authorize, document, and deconflict tasks to be carried out by frontline workers. A PTW system has the goal to reduce unsafe activities in non-trivial work environments.

Oil rig structures, maintenance, and permitting is shown in 4D view using LuciadLightspeed. A detailed oil rig 3D CAD model is shown with renderings that reveal the complete structure. By combining the timeline view of distribution of work permits with the 3D model, permits are immediately visualized for the selected date and time. Then, selecting a permit highlights the relevant parts of the entire structure in the 3D model.

16

Link to demonstration movie: 4D Visual Work Permit Analysis of Oil Rig

Visually Analyzing Millions of Twitter Feeds

Millions of Twitter feeds are visually analyzed in this collaboration of Hexagon’s Geospatial division, Hewlett Packard Enterprise, and Dataiku. The Twitter feeds are extracted from HP Vertica Database and analyzed by Dataiku Data Science software. By combining the clustered tweets, a basemap, a timeline, and filtering of topics in LuciadRIA, this application provides interactive analysis through instant visual updates.

Link to demonstration video: Twitter Feed Analytics Using LuciadRIA, HP Enterprise, and Dataiku

17

Contact Us

https://go.hexagongeospatial.com/contact-us-today

About Hexagon Hexagon is a global leader in digital solutions that create Autonomous Connected Ecosystems (ACE). Our industry-specific solutions create Smart Digital Realities™ that improve productivity and quality across manufacturing, infrastructure, safety and mobility applications. Hexagon's Geospatial division creates solutions that visualize location intelligence. From the desktop to the browser to the edge, we create ACE that bridge the divide between the geospatial and the operational worlds. Hexagon (Nasdaq Stockholm: HEXA B) has approximately 20,000 employees in 50 countries and net sales of approximately 4.3bn EUR. Learn more at hexagon.com and follow us @HexagonAB. © 2019 Hexagon AB and/or its subsidiaries and affiliates. All rights reserved. Hexagon and the Hexagon logo are registered trademarks of Hexagon AB or its subsidiaries. All other trademarks or service marks used herein are property of their respective owners. Hexagon’s Geospatial Division believes the information in this publication is accurate as of its publication date. Such information is subject to change without notice.