NetLens:Iterative Exploration of Content-Actor Network Data

Hyunmo Kang, Catherine Plaisant, Ben Bederson

Challenges of Network Data Visualization

by Frank van HamTouchGraph

Challenges of Network Data Visualization

Complex Analytic Tasks Incremental data exploration Iterative query refinement

Scalability Common simple UI components

e.g. histogram and lists

Generality Apply to any dataset matching Content-Actor model

e.g. digital library, photo collections, email collections, case law, etc.

Data Analysis (Content-Actor Model)

Entity E1(Content)

Entity E2(Actor)

Intra-relationship Intra-relationship

Inter-relationship

Data Analysis (Content-Actor Model)

Content(Paper)

Actor(Author)

Intra-relationship Intra-relationship

Inter-relationship

Paper

Year

Reference People

Title

Category Keywords Address

Name

Author

Data Analysis (Content-Actor Model)

Content(Email)

Actor(People)

Intra-relationship Intra-relationship

Inter-relationship

Email

Date

Reply, Forward

People

Subject

Size Attachment Address

NameTo, CC

From

Data Analysis (Content-Actor Model)

Actor(Photo)

Content(People or Category)

Intra-relationship Intra-relationship

Inter-relationship

Photo

DatePeople

Comments

Path Favorite

X, Y location

X, Y location

HAS

HAS Category

DEMOScreenshots at www.cs.umd.edu/hcil/netlens Video at www.cs.umd.edu/hcil/netlens/VASTvideo

Task Analysis

Search By

SearchResult

Entity1 (Paper) Entity2 (People)

Entity1’s Attributes Entity1(Paper) Entity2’s Attributes Entity2 (People)

Entity1(Paper)

Search for papers by paper attributes such as year, keywords, title, conference, topics, etc.

Search for papers that “cite” or “are cited by” the selected papers along with frequency

Search for papers by people attributes such as author’s affiliation, institution, nationality, etc.

Search for papers written by the selected authors (either conjunctive or disjunctive)

Entity2(People)

Search for authors by paper attributes such as year, keywords, title, conference, topics, etc.

Search for authors of the selected papers with frequency (the number of papers per each author)

Search for authors by people attributes such as author’s affiliation, institution, nationality, etc.

Search for academic advisors of the selected authors (either conjunctive or disjunctive)

Task Analysis

Single step tasks How many papers on “User Study” were published in 1998? Who are the authors of the papers on “Virtual Reality”, which

were published at the CHI 99 conference? Which paper is the most frequently cited by the papers

published at the CHI 04 conference? Which author is most frequently cited in the “InfoVis” topic? How many papers were published by UMD HCIL people? Who are the authors whose nationality is Korea?

Task Analysis

Multiple step tasks Evaluate individuals:

- how many papers were self-referenced? - how frequently was each paper referenced by other papers?

Identify communities:- what are the major paper topics published by UMD HCIL? and who in this group has the most papers in that topic?- how do UMD HCIL’s research interests change over time? and who in this group made that change?

Find experts (to review papers or come to workshop):- who wrote the most papers in the InfoVis topic? and how many papers cited his papers?- whose paper in the InfoVis area is most frequently referenced by other papers?

Learning about a new topic (to find a good PhD topic):- which topic has growing publications? and who contributed most to this topic last 3 years?- what are the other topics the authors in InfoVis area also get interested in?

Where should I go on a sabbatical?- which country (or research group)’s authors most frequently reference my papers?

Design Challenges

History and Integrated HelpSequence of interactions to

accomplish a task (lost in exploration)

“How did I get here?”“What does the current filtered

dataset mean?”

Design Challenges

Multi-layered InterfaceUsers do not need all the windows

• Complexity of data and tasks

• Computation efficiency

• Users’ usage levels and their preferences

• Etc.

Design Challenges

Data ExportIntegration of graph visualizing tool

TreePlusExporting methods

• Windows clipboard

• Internal graph class object

• Xml documents

NetLens Extension

Emails on the leftPeople on the right

Overviews provided for all attributes (here for emotional tone on emails side)

Filtered to show only emails related to CAenergy crisis; and the people who sentthem are shown on the right side.

(Joint Institute for Knowledge Discovery) - http://jikd.umiacs.umd.edu

NetLens Extension(Joint Institute for Knowledge Discovery)

• Generality and Scalability

JIKD data schema

NetLens Data Schema

NetLens System Architecture

NetLens Written in C#, Piccolo toolkit

Database Server MySQL

ADO.NET driver for MySQL MySQL connector/NET 1.0.7

Web Server Mac OS X Server

Web API CGI, JSP (e.g. email search, people’s bio, etc.)

Evaluation

Heuristic Evaluation by NIST Possible directions:

Usability• Measure usability• Speed, performance, • Learnability• Error rates

Power• Comparing range and complexity of possible

queries• SQL queries?

Generality• How easy it is to apply new datasets to NetLens

PhotoMesaBrowse, Annotate, and Search Digital Images

Hyunmo Kang and Ben Bederson

PhotoMesa Image Browsing

Zoomable User Interface

Zooming into a group of photos

Zooming into a single photo• PhotoMesa shows all photos in a single view• Bigger preview by moving over a thumbnail• Browse photos by zooming in or out• Dynamic sorting and grouping

PhotoMesa Image Browsing

Zoomable User Interface

• PhotoMesa lets you control visible photos

• All photos• Unhidden photos• Representative photos• Favorite photos only

Show only the representative photos for each group

PhotoMesa Image Browsing

Zoomable User Interface

• Browse photos in “Scroll” mode with detail photo view

Photo Information with EXIF

Scrollable Thumbnails Panel

Detail Photo Panel

PhotoMesa Annotating

Add a caption and mark photo as favorite or hidden

Label who is in the photo

Label objects in the photo (e.g. animals, locations, etc.)

• People Annotation Mechanisms: Checkbox Annotation, Drag-and-drop Annotation, Hotkey Annotation

• Category Annotation: Create user-defined hierarchical structure of object types to annotate your photos with

PhotoMesa Annotating

• Bulk Annotation: Annotate multiple photos simultaneously with the same annotation mechanisms

PhotoMesa Searching

By keyword

By folders

By people

By category

By year

By month

PhotoMesa Photo Sharing

Upload Photos Metadata e.g.) people, category, photo info, etc.

Remove Update Search Web Services

Browse with web browser Add comments

PhotoMesa Data Schema

PhotoMesa SQL Query

Free Text Search (Find photos containing word “kang”)

SELECT Photos.*FROM (Photos INNER JOIN (Categories INNER JOIN

PhotosCategories ON Categories.categoryname = PhotosCategories.categoryname) ON Photos.url = PhotosCategories.url) INNER JOIN (People INNER JOIN PhotosPeople ON People.personname = PhotosPeople.personname) ON Photos.url = PhotosPeople.url

WHERE (((PhotosPeople.personname) Like "*kang*") OR ((Photos.url) Like "*kang*") OR ((Photos.created) Like "*kang*") OR ((Photos.uploaded) Like "*kang*") OR ((Photos.description) Like "*kang*")) OR (((PhotosCategories.categoryname) Like "*kang*"));

PhotoMesa SQL Query

Add Photo

INSERT INTO Photos (url, created, uploaded, description, photomark, thumbnail, width, height) Values(“url”, “2006-04-20", “2006-04-20", “hyunmo’s trip to Seoul”, 1280, 1024);

Add People

INSERT INTO PhotosPeople (url, personname, x, y, time) Values(“url”, “hyunmo kang“, “0.1234”, “0.789” “2006-04-20");

INSERT INTO People (personname, lastname, firstname) Values(“hyunmo kang”, “kang”, “hyunmo”);

PhotoMesa System Architecture

PhotoMesa Client Written in C#, Piccolo toolkit

Database Server MySQL

ADO.NET driver for MySQL MySQL connector/NET 1.0.7

Web Server Apache

Web API PHP (photo upload, web services)

Questions?

More visualization projects are available athttp://www.cs.umd.edu/hcil

PhotoMesahttp://windsorinterfaces.com

NetLenshttp://www.cs.umd.edu/hcil/netlens

Email: kang@cs.umd.edu