multimedia information systems vu...

Visual Analytics 07 Dec 2015Vedran Sabol (KTI, TU Graz)

Visualisation in the Web

Multimedia Information Systems VU (707.020)

Vedran Sabol

KTI, TU Graz

7th Decmber 2015


Overview

• Motivation and Definition

• Introduction to Visualization

• Visualization examples and demos

Document content

Multidimensional data

Structures

Temporal and geospatial data

Multiple-visualisation interfaces

• Visual Analytics

• Web Visualisation Technologies and Frameworks

2


Visualisation - Motivation

• We are confronted with:

Massive amounts of information

Complex heterogeneous information

Dynamically changing data

Uncertain, incomplete and conflicting information

…

• Big Data

• The four V’s: Volume, Variety, Velocity, Veracity

• Difficult to process using traditional applications and methods

3



How can computers help us to understand data?

Interactive exploration and analysis of data

Unveiling facts and knowledge hidden within the data

4


Knowledge Discovery Process

• Knowledge Discovery Process [Fayyad, 1996]

Mainly an automatic approach consisting of a chain of processing steps

Goal: discovery of new, relevant, previously unknown patterns and relationships in data

5

FeedbackTarget Data

Transformed

Data

Patterns &

Models

Preprocessed

Data

Data

USER

Knowledge

Preprocessing & Cleaning

Data Transformation

Data Mining & Pattern Discovery

Interpretation & Evaluation

Data Selection



• Machines are very powerful

Automatic processing methods for huge data sets

Exponential growth of computer-performance since 60 years

• Moor‘s Law: continues until 2020, 2030… ?

Distributed computing: Cloud, Grid, …

• Nevertheless, machines still behind humans in

Identification of complex patterns and relationships

Wide knowledge, experience, intuition

Abstract thinking

…

6



• Human visual apparatus is an extremely efficient „processing machine“

• Enormous amounts of information are transferred by the visual nerve into the brain cortex

Extremely high bandwidth

• Visual cortex remains unbeatable in recognition of objects and complex patterns (e.g. rotational invariance)

• Pre-attentive processing

7


Pre-attentive Processing

• Capability to process certain visual information without focusing our attention

• Criterion 1: Processing time < 200 - 250ms

Eye movements in about 200ms single glimpse

Highly parallel processing

• Criterion 2: Processing time does not correlate with the amount of noise in the data

• Limited number of pre-attentive features

Size (length/width), number, colour, intensity, curvature, orientation, flicker, motion direction, 3D depth-cues…

8



It is immediately possible to determine which data set contains a red spot Pre-attentive processing possible

9



It is still possible to quickly determine where the red spot is Borderline case

10



Scanning is necessary to determine where the red spot is Pre-attentive processing not possible

11


Visualization – Approach and Challenges

• Approach

Machines transform the data into a suitable graphical representation

Employ the human visual system for pattern recognition

• Challenges

How should the graphical representations look like (design)?

How to compute the graphical representation (algorithms)?

Which operations shall be supported on the graphical representation (interactivity)?

12


Visualization - Definitions

Graphical representation of data, information and knowledge

Use of human visual system, supported by computer graphics, to analyze and interpret large amounts of data

The use of visual representation to aid cognition

“Transformation of the symbolic into the geometric.”[McCormick et al., 1987]

“The depiction of information using spatial or graphical representations to facilitate comparison, pattern recognition, change detection, and other cognitive skills by making use of the visual system. “ [Hearst, 2003]

…

13


Visualization - Examples

14


Visualization DesignRepresentation Forms

• Fundamental categories of visual representation:

Formalisms

Metaphors

Models

• Formalisms: abstract schematic representations

Defined by a designer

Users must learn how to read and interpret

Example: Percentage is represented by an arc

15


Visualization DesignRepresentation Forms

• Metaphors: representations based on a real-world equivalent

Intuitive

User can understand the meaning through building analogies

Example: using the geographic map metaphor to represent similarity in non-spatial data

• Models: based on mental representations of the physical world

Data typically has a natural representation in the real world

Examples: visualization of sensory data in 3D, virtual 3D worlds

16


VisualizationData – Information - Knowledge

• Data/Scientific Visualization

• Information Visualization

• Knowledge Visualization

17

Data KnowledgeInformation

Representation complexity, applicability by humans

Machine processing capability



• Data

Formal representation of raw, basic facts

Have a fixed format: numbers, dates, strings,…

Have a fixed, predefined meaning (i.e. no interpretation required)

„3162“ – Hotel room number (not a telephone number)

• Information

Result of processing, manipulation and interpretation of data

May not have a fixed format (unstructured or semi-structured)

Meaning is determined by interpretation within some context

“A small mouse” – a computer or a field mouse? (determined by context)

18



• Knowledge

Identified, organized and as valid recognized information

Representations of reality through abstract, domain-dependent models

Represented by formalized conceptual systems: Taxonomies, Thesauri…

Ontologies are formally defined knowledge representations consisting of concepts, relations and rules (axioms)

19

Animal

Mouse is aLegs

has

Jerry is a

Visual Analytics 24 Nov 2014Vedran Sabol (KTI, TU Graz)

VisualizationSubdivision

• Data/Scientific Visualization

• Sensor data

• 3D spaces

• Knowledge Visualization

• Knowledge models, knowledge transfer

• Information Visualization

• Document content (text and multi-media)

• Multidimensional data sets

• Structures: hierarchies and networks (graphs)

• Temporal information

• Geo-spatial information

20


Data/Scientific Visualization

• Visualization of simulation or sensory data

have a natural representation in the real, physical world

• Applications in physics, medicine, astronomy, automotive…

• Web technologies: HTML5 canvas/WebGL

21

Pressure coefficients [NASA] Coil magnetic field


Knowledge Visualization

• Knowledge Visualization is about using visual representations to present and transfer existing (explicit) knowledge between people [Eppler]

• The focus is on structured knowledge spaces

Concepts, relations, facts, attributes

Navigation along structures present in the knowledge model

• Use of metaphors and formalisms

22


Knowledge VisualizationExamples

23

Stairs of Visualisation [Eppler](Let‘s Focus: http://en.lets-focus.com/ )

Research Map [Bresciani]

http://en.lets-focus.com/



Gyro, Know-Center [Kienreich]

24

Cultural Heritage Visualization Ancient Theatres [Blaise]



• Gyro: visualisation of Brockhaus encyclopedia articles

Viewed article in the center

Related articles around it

Icons encode article type (person article, geo-location article etc.)

• Cultural Heritage Visualization

Roman amphitheaters in France

Uses formalisms to describe characteristics and state

25


Information Visualization

• Interactive visualization of abstract information spaces

Abstract information has no „natural“, real-world representation

Rely on metaphors and formalisms

• Goal: identifying patterns and relationships

Explorative analysis and navigation

Unveiling of implicit knowledge

• InfoVis Mantra [B. Shneiderman]

„overview first - zoom and filter - details on demand”

26


Visualization Examples Document Content Summary

MovieDNA [Ponceleon]TileBars [Hearst]TagClouds, Know-Center [Seifert]

27


• Tag Cloud

• Keyword visualisation

• Size corresponds to importance of the keyword

• TileBars

• Shows where keywords/concepts occur in the document

• Darker color – more occurrences at a given position

• Easy to find interesting parts in large documents

• MovieDNA

• Analogous concepts for videos

28

Visualization Examples Document Content Summary


Visualization Examples Multidimensional Data

Scatterplot [Nowell] Parallel Coordinates [Inselberg]

29


• Multidimensional data

• Each data item described by a large number of features

• Scatteplot

• Data elements represented by dots/icons

• Up to five dimensions

• Using 2 axes (x and y) and icon shape, color and size

• Parallel Coordinates

• Data elements represented by lines

• Each dimension is a separate y axis (many are possible)

• Line color an additional dimension

• Filtering along different dimensions

30

Visualization Examples Multidimensional Data


Visualization Examples Multidimensional Data Similarity - Text

Know-Center [Sabol et al.]

Galaxies (SPIRE), PNNL [Wise]

31


• Example: unstructured data - text

• Text analysis to transform text to multidimensional data

• Dimension are words/concepts in the text

• Makes documents topically comparable

• Information landscape (metaphor)

• Provides a topical overview of a (large) document repository

• Encodes topical similarity through spatial proximity

• Spatial clusters (hills) represent group of documents with similar topics

• Visualises cluster distribution, outliers

• Automatically extracted labels specify topical regions of the info-

landscape

32

Visualization Examples Multidimensional Data Similarity


Visualization Examples Multidimensional Data Similarity - Images

Image Similarity Layouts [Rodden]

33

• 3 Layouts:

• Overlapping, non-overlapping, space-filling


Visualization Examples Hierarchies

TreeMaps [Shneiderman]Hyperbolic Tree (InXight) [Lamping]

34


• Hyperbolic Tree

• Large hierarchies shown in hyperbolic space

• Information at the edges geometrically “compressed”

• Tree branches not in the focus are small but remain visible

• TreeMap

• Hierarchy represented by nested rectangles

• Size and color of a rectangle encode properties

35




36

InfoSky, Know-Center [Andrews et al.]

Circle Packing, D3 library


• InfoSky

• Visualisation of large, hierarchically organised document collections

• Hierarchy shown as nested Voronoi areas

• Documents shown as dots

• Distance in the visualisation represents topical similarity

• Color-coding used to represent document properties

• Interactive navigation in deep, complex hierarchies

• Packed Circles

• Similar approach using nested circles

• Disadvantage (vs. InfoSky): screen area between circles is wasted

37



Visualization Examples Graphs

38

Gephy, https://gephi.org/

Narcissus (3D) [Hendley]


• Node-link diagrams

• Nodes represent entities

• Links represent relationships between them

• Properties encoded by colour, icon, size/thickness

• Problem: link clutter for large number of relationships

• Graph layout methods

• Large, complex area of research

• Popular: force-directed placement

– Connected nodes exert attractive forces (spring model)

– Good for small graphs (not scalable)

– Does not solve the link clutter problem

39

Visualization Examples Graphs


Visualization Examples Graphs – Edge Bundling

40

Edge-Bundling [Holten & van Wijk]

Concept Networks [Kienreich] (Know-Center)


• Edge Bundling reduces clutter enormously

• Bundle together edges that

• Propagate in the same direction

• Are close to each other

• Are of comparable length

• Overlap

41

Visualization Examples Graphs – Edge Bundling


Visualization Examples Temporal Data

42

LifeLines [Plaisant]Themeriver, PNNL [Havre]


• Timeline / LifeLines

• Displays event or time intervals in chronological order

• Time flow along an axis

• LifeLines: application for patient histories

• ThemeRiver (StreamGraph)

• Thematic variations (trends) across several categories

• With of a “stream” represents the strength of the topical category at a

given time point

43




44

Spiral geometry [Carlis] Perspective Wall [Mackinlay]


• Spiral geometry timeline

• Longer time intervals possible than with the classical timelines

• Suitable for periodic event visualisation

• Perspective Wall

• A timeline variant with 3D “distortion”

• Suitable for large time intervals

• Only the time interval in the focus is displayed in high-detail

45



Visualization Examples Spatial Data

46

LucentVision [Pingali 2001]

Planetarium, Know-Center [Kienreich]


Visualization Examples Geospatial Data

47

Google Maps

APA-Labs component, by Know-Center


• Variable level of detail(LOD)

• Technique known from 3D environments

• Decrease complexity of representation for “far-away” objects

• Coarse-grained view of the whole data space

• Provide more details when zooming-in

48

Visualization Examples Geovisualisation


Geovisualisation with Variable LODGoogle Maps

49


Graph Visualisation with Variable LOD Mouse Anatomy Ontology (Overview)

50


Graph Visualisation with Variable LOD Mouse Anatomy Ontology (zoomed in)

51


Visualization ExamplesMultiple Data Aspects – Geo-Temporal

GeoTime, Oculus [Kapler]

52


• Oculus GeoTime

• Combines geo-visualisation and a timeline in a 3D view

• Geovisualisation: x-y plane

• Time: z-axis

• Follow movements along space and time

• Easy to identify encounters/meetings

53

Visualization Examples Multiple Data Aspects – Geo-Temporal


Visualization ExamplesMultiple Data Aspects – Visual Links in 3D Environments

Starlight, PNNL [Risch et al.]

54

Caleydo, ICG, TU Graz [Lex et al.]


• Visual Links

• Multiple visualisations showing different aspects of the data

• Shown in a 3D space

• Represent links between data elements in different visualisations

55

Visualization Examples Multiple Data Aspects – Visual Links in 3D Environments


Visualization ExamplesMultiple Data Aspects – Coordinated Multiple Views

Coordinated Multiple Views

• Multiple visualizations “fused” into a single, coherent user interface

• Each visualization designed to convey a different aspect of the data

simultaneous navigation and analysis over multiple data aspects becomes possible

• Coordination of state and behavior

• interactions in one visual component influence all others

• Selection, filtering, visual properties, navigation, …

56



Spotfire [Schneiderman]

57



58

• Spotfire

One of the first commercial CMV-enabled InfoVis tools

Introduced coordination on database level (“Snap Together”)

• CODE Visualisation Wizard

Coordinated visualisation of linked data

Utilises semantic information to link data sets

Developed at Know-Center

http://code.know-center.tugraz.at/vis


Coordinated Multiple ViewsCODE Visualisation Wizard

59


Coordinated Multiple ViewsCODE Visualisation Wizard

60


Visual Analytics

61

• Motivation

• Machines and humans have complementary capabilities

• Abundance of data: problems too large to be addressed by visualization alone

• Limited resources of the visual front end

• Idea: combine machine processing with human capabilities in a suitable way and get the best of both worlds.

• Integrate humans in the analytical process

• Provide means for explorative analysis

• Let machines learn from humans

• Visual Analytics: a young research field (2005)


Visual Analytics

62

• Combines automatic methods with interactive information/data/knowledge visualisation to get the best of both worlds [Keim 2008]

• Focuses on interaction between humans and machines through visual interfaces to derive new knowledge

• Supports analytical reasoning facilitated by interactive visual interfaces [Thomas 2005]

Repository

New Insights and Knowledge

Algorithms Visualization


Visual Analytics

63

• Main Idea (Mantra): “analyse first – show the important – zoom, filter and analyse further – details on demand” [Keim 2008]

Initial analysis and visual pattern recognition

Posing a hypothesis

Further analysis steps (automatic and interactive)

Confirmation or rejection of the hypothesis: new facts

Confirm the expected, discover the unexpected

• Challenges [Keim 2009]

Balance between automatic and interactive analysis

Design of effective VA workflows

Scalability



64

• AJAX lays the foundations – asynchronous requests

• Rich, responsive user interfaces

• HTML5 provides the basis for visualization in the Web

• New APIs

• Drag-and-drop

• WebWorkers – background processing

• Storage (FileSystem API, IndexedDB…)

• Cacheing (enables Web Apps)



65

• Rendering

• Canvas

• SVG

• WebGL

• Logic and Interactivity

• JavaScript: high-performance engines (compile to native)

• Server-Client Web architectures fit the needs of Visual Analytics

• Model View Controller (MVC) architecture

• Data storage/crunching on the server

• WebSockets – bidirectional server-client communication


Web Visualization Performance and Scalability

• Scalability limited by computing power of the client

SVG: hundreds of items, easy to program

Canvas: at least one order of magnitude better

WebGL: potentially millions of items

• How to scale to large (huge) data sets

Millions (or billions) of data elements

Utilise the power of the server (Visual Analytics)

66


Web Visualization Toolkits and Rendering Libraries

• General visualisation

D3.js: "Data-Driven Documents" JavaScript visualization library libraryusing SVG

JavaScript InfoVis Toolkit: for creating Interactive Data Visualizations

• Drawing and rendering

Raphaël: JavaScript library for vector graphics

Paper.js: open source vector graphics scripting framework that runs on top of the HTML5 Canvas

EASELJS: makes working with HTML5 canvas easy (Flash-like API)

67



• Charting and plotting

NVD3: Re-usable charts for d3.js

gRaphaël: JavaScript library for creating charts

jqPlot: a plotting and charting plugin for the jQuery Javascriptframework

Flot: JavaScript plotting library for jQuery

Flotr2: JavaScript library for drawing HTML5 charts and graphs

Peity: jQuery plugin for converting element's content into a mini-chart

Google Chart Tools (free usage, but NOT OPEN SOURCE)

RGraph: free (for NON-COMMERCIAL use only) HTML5 charts

Highcharts JS: free (for NON-COMMERCIAL use only) javaScript chartslibrary

68



• Graphs and trees

arbor.js: a graph visualization library using web workers and jQuery

sigma.js: lightweight JavaScript library to draw graphs using HTML canvas

jsPhyloSVG open-source javascript library for rendering phylogenetic trees

GraphGL: a network visualization library for massive graphs based on WebGL and WebWorkers

• Maps and geovisualisation

Leaflet: Open-Source JavaScript Library for Mobile-Friendly Interactive Maps

Kartograph: lightweight framework for building interactive mapapplications

Polymaps: JavaScript library for image- and vector-tiled maps usingSVG

69



• Time series and temporal data

Rickshaw: D3-based JavaScript toolkit for creating interactive time series graphs

SIMILE Widgets: Open-Source Data Visualization Web Widgets

Cubism.js: D3 plugin for visualizing time series

dygraphs: open source JavaScript library for time series charts

Envision.js: library for creating interactive HTML5 visualizations

Timeline JS: beautiful, easy to use timelines

70



• 3D

three.js: WebGL rendering back end with optional fallback on canvasor SVG

LibGdx: Java game development framework, compiles to all desktopand mobile platforms, including to JavaScript/Browser

• Multiple view coordination

Crossfilter: Fast Multidimensional Filtering for Coordinated Views

• Programming languages and environments

Processing.js: visual programming language and environment designedfor the web

71