Uncertainty Computation,Visualization, and Validation

Suresh K. LodhaComputer Science

University of California, Santa Cruzlodha@cse.ucsc.edu

(831)-459-3773

Personnel

• Suresh K. Lodha– PhD, CS, Rice University, 1992– Scientific & information visualization,

uncertainty quantification & visualization,multi-modal visualization,computer-aided geometric design

– Uncertainty research supported byNational Science Foundation &Department of Energy ASCI Program

(LLNL, LANL, SNL)

Overview

• Uncertainty representation & computation• Data/information fusion• Quality-of-service issues• Uncertainty visualization• Uncertainty validation

Uncertainty Visualization Pipeline

Physicalphenomenon

Informationacquisition

-sensors -humans -databases

Uncertainty computation

Uncertaintyvisualization

-target deformation -glyphs -animation

Info. processing& fusion

-computational algorithms

Analysis

-interaction -decision

Transformation

-sampling -quantization -compression

Sources of Uncertainty

• Sensor and human limitations• Noise, clutter, jamming etc.• Modeling assumptions• Algorithm limitations• Data compression• Communication errors• Visualization-induced errors

Uncertainty Representation

• Uncertainty formalisms used by the fusion community– Probability– Dempster-Shafer evidence theory– Fuzzy sets and possibility theory

• Uncertainty representation in visualization research – Confidence intervals– Estimation error– Uncertainty range

Uncertainty Computation(Previous Work)

• Data/Information Fusion– Knowledge-based systems– Random sets (Goodman, Nguyen, Mahler)

• Visualization– NIST/ NCGIA `91 (Beard et al.)– BattleSpace `98 (Durbin et al.)– Visualization Software `96 (Globus, Uselton)– Scientific Visualization `96 -- (Lodha, Pang,

Wittenbrink)

“Any battlefield necessarily deals with uncertainty, and it is necessary to determine ways to represent and encode the confidence level that exists for each piece of battlefield data.” – Durbin et al ’98 (NRL)

Designing Error Metrics

• True vs. measured/observed/anticipated• Observed vs. simulated• High resolution vs. low resolution• Continuous vs. discrete• Individual source vs. multiple sources• Static vs. dynamic• Time-independent vs. time-critical• Error-free vs. error-prone communication

Examples of Error Metrics

• Local metrics -- distance metric-- curvature metric-- sampling-number or depth metric

(distribution of error)• Global metrics

-- Topology metric

Uncertainty Metrics : Isosurfaces

Uncertainty Metrics : Fluid Flow Topology

Original

332 cp65%

55%

Research Issues

• Representations and data structures for uncertainty measures

• Design and integration of error metrics• Uncertainty-aware and uncertainty-reducing

data processing (algorithms and models)• Common consistent uncertainty

representation over a distributed mobile network ?

Uncertainty Visualization

• How to convey uncertainty to human users?– Uncluttered display– Intuitive metaphors for mapping– Data characteristics– Multi-modality

• Do NOT hide processes that produce problems for the human users?– Visualize the abstraction (e.g uncertainty

pipeline, graphical models)

Uncertainty Visualization

• Display devices /environment– screen space (monitor, PDA, workbench,..)– mobility

• Modality– vision– audio– haptics

Uncertainty Visualization

• Data types/ characteristics– scalar/vector/tensor– discrete/continuous– static/dynamic

• Levels of fusion– data-level (raw/abstract)– image-level (physical phenomena)– feature-level (compressed view)– decision-level (super-compressed)

Uncertainty Visualization (continued)

• Techniques– glyphs– deformation– transparency– texture– linking– superimposing/backgrounding– augmented reality– modality

Unc Viz: Example 3Fluid Flow Visualization

Unc Viz: Example 5 Geometric Uncertainty

Uncertainty Visualization: Example 4

Research Issues

• Uncertainty mapping and metaphors for different modalities, data types and fusion tasks

• Display support for a variety of uncertainty metrics/formalisms

• Interactive display for uncertainty-source -> task analysis

• Integration and analysis of uncertainty for decision-making?

Uncertainty Validation

• Does addition of uncertainty information help human users in making decisions?

• Can humans integrate qualitative and quantitative (or heterogeneous information) when there is uncertainty?– Task definitions– Careful design of experiments– Usability studies– Statistical analysis

Uncertainty Validation

• Task definitions– primary level tasks (raw estimation)– secondary level tasks (correlation or simple spatio-

temporal relationships)– higher level tasks

• Examples– feature existence (binary decision)– feature recognition (finite multiple choices)– target aiming (zone-centered decision within a

specified space-time region)

Validation Strategies

• Formative vs. summative studies• With or without uncertainty mapping• Representative sampling of tasks, data and

uncertainty mappings• Constrained, interactive or free-form

environment• Within-subjects/between-subjects and

tabular designs

Uncertainty Validation (Previous Work)

• Validation of user interfaces (CHI `90s) • Validation of multi-modal mappings

(Melara, Marks, Massaro (UCSC))• Validation of uncertainty mappings

Uncertainty Validation: Example 1 (with M. Hansen)

• Protein structural alignment (intuitive metaphors)

Uncertainty Validation: Example 1 (continued)

• Protein structural alignment-- accuracy of discrimination

Uncertainty Validation: Example 2 (GIS)

Rainbow Saturated

Uncertainty Validation:Tasks(averaging, comparisons)

Uncertainty Validation: (with Wittenbrink & Pang)

• Vector uncertainty glyph evaluation

Research Issues

• Construction of user evaluation environments

• Conduct user evaluation studies for efficiency and accuracy

• Data analysis and statistical testing• Feedback loop to improve performance• Integrated decision tool combining

uncertainty approaches in visualization and command and control?

Concluding Remarks I

• Provide human users with uncertainty information– Representation and computation of uncertainty– Uncertainty-aware and uncertainty-reducing

algorithms and models– Uncertainty visualization– Visualization of uncertainty pipeline or hidden

processes or abstract models

» (continued)

Concluding Remarks II

• Effective and clutter-free visualization of uncertainty along with the data/information– Sensitive to data characteristics/ fusion level/

tasks/ display environments (intuitive and cognitively accurate metaphors)

– Multi-modality – Usability studies

Collaboration

• Uncertainty representation/ fusion(UCSC, Syracuse GTech, UCB,USC)

• Uncertainty visualization(UCSC, Gtech UCB, USC)

• Multi-modal interaction(UCSC USC, GTech)

• Other MURIS/ DoD?