designing for collaboration: challenges & considerations of multi-use information visualization...

Designing for Collaboration
Challenges and Considerations of Multi-Use Information Visualization Tools

Stephanie Gokhman
April 8, 2011Allen Library

Introduction:
I'm Stephanie Gokhman first year PhD student jointly affiliated with the Communicative Practices Lab with Mark Zachry and the Computer-Supported Collaboration Lab with Charlotte Lee. The research presented here is a small piece of a multi-year NSF-funded project with Mark and David McDonald from the iSchool.

Overview

Introduction: What is Re:Flex? Why Wikipedia?

Literature Review: What do we know about building systems to support collaboration through visualizations?

Methods: How do we expose the considerations in designing information visualization tools for collaboration?

Results & Discussion: What did we find and how does this give us insight into collaborative information visualization tools?

Future Work

This research is just one of many stages and offshoots of that funded initiative. We are developing an information visualization toolset for use by Wikipedians and for use by us to better understand the collaborative effort that creates Wikipedia. We call this Re:Flex, which stands for Reflexive Flexible Reputation tool. It also supports a number of other verbs with RE and FLEX (such as REminisce, REtrospection, etc). We'll briefly discuss the greater overall Re:Flex project and why we selected Wikipedia later, but this particular talk is focused on foundational work to inform the information visualizations that comprise the toolset...




Information Visualizations to Support Collaboration

What are the steps to appropriately design information visualization tools to support collaboration?

The need for these tools is apparent but the method of designing them is not. Therefore this research was intended to answer the question What are the steps to appropriately design information visualization tools to support collaboration?

Re:Flex:
The Big Picture

Communicative Practices in Virtual Workspaces Lab

Set of tools to expose and visualize the relationships between editors and editors and editors and articles on Wikipedia

Focused on the types of work present in the Wikipedia community

Help Wikipedians recognize and value work in the Wikipedia community

Re:Flex is a reflexive, flexible reputation toolset using information visualizations to expose relationships between editors and other editors as well as editors and articles, or namespaces

Through these visualizations we hope to understand how to build systems to support collaborative spaces by exposing and visualizing relationship data. This is referred to as social translucence.

Why Wikipedia?

Largest freely accessible online collaboration

Largest, most popular reference work on the Internet, according to Alexa Statistics1

Characteristics of other collaborative spaces which these visualizations displayRelationshipsEdit historiesConflictUser pagesNegotiation

Multiplicity of potential spaces where information visualization would serve as meaningful tools

We selected Wikipedia to begin this research because it is the largest online collaboration, with all data since it's inception freely accessible, and contains a multiplicity of collaborative activities that are common across multiple contributor systems.

Say the sentences put a picture of the homepage

The BIGGER Picture:
Beyond Wikipedia

Allow large data sets, specifically edit histories, to be explored by many contributors collectively and individually

view data in context

observe patterns

make comparisons through visualizations

further understanding the community that has generated the body of knowledge

The visualizations are being designed with broader applicability to contributor systems in mind. These are systems in which public engagement provides content, where there exist user pages and edit histories as markers of contribution.

When building Re:Flex we determined components of collaboration in these types of systems that we viewed as most meaningful and best supported by information visualizations: viewing data in context, observing patterns, making comparisons through visualizations by synthesizing the large data sets and generally better understanding the community that has generated the body of knowledge.

Beyond Wikipedia

Underlying principles of Re:Flex serve a basic need across a number of collaborative technologies

Could be applied to other popular contributor systems: Quora1

GitHub2

TED Conversations3

1. http://www.quora.com2. https://www.github.com3. http://www.ted.com/conversations

These are a series of examples of contributor systems which also provide a context in which support for those activities would be meaningful.

Quora is a collection of publicly created questions and answers, sometimes referred to as a knowledge market for it's available knowledge resources.

GitHub is a platform that hosts open source coding projects where public participants edit and add to applications.

TED Conversations is platform similar to Quora, yet is not restricted to questions an answers. This platform mimics what some of us would see as a messageboard style platform and dynamic.

Information Visualizations
to Support Collaboration

As collaborative technologies increase in attention and prevalence, there exists a blossoming design space for information visualization tools built for collaborative environments

The complexity of collaborative spaces reveals a large number of tensions within this design space that have not been fully exploredWhat are the steps to appropriately design information visualization tools for collaboration?


Informing the Design of Re:Flex

What are the salient activity types?

What are the fundamental data types?

What are the biases of the users of Wikipedia?

What relationships do the users want to see visualized?

What relationships do the users need to see visualized?

Methodology

Interview, Wants/Needs analysisInformed visualizations built by a graphic designer from the Information School

Surveyed 100+ Wikipedians

Visual comparisons of the same data or problem space to compare two different designs on the same issue

Represented the multiple dimensions of value and different kinds of data, specifically focused on addressing a contributor as a valued member of the community or a vandal

To answer this question, we employed a jointly qualitative and quantitative approach to better understand the types of data the community values, the types of visualizations the community finds preferable, and inherent social biases and constraints.This data collection began prior to my involvement in the project by Elly Searle. We invited Wikipedians from the Puget Sound region (from Portland to Vancouver, BC) to deeply understand their individual usage and community behaviors.By conducting a wants/needs analysis on a series of interviews with active Wikipedians she selected a number of primary interaction/activity types and fundamental data types which inform these activities. Using this information we hired a graphic designer from the User-Centered Design program to create a series of visualizations which we then tested through a survey of over 100 active Wikipedians.

Synthesis of Interviews:
Uncovering Salient Data and Action Types

Actions:Content creation

Vandal-fighting

Quality control

Reversion

Appreciation

Data Types:Editor

Total Number of edits by editor

Articles

Barnstars

Namespaces

The above list is the result of a second pass at the interview materials from the wants/needs analysis. These are the activity types and fundamental data points that we are using to inform the current iteration of visualization design.

Graphical Test Bed

Visual metaphor to convey relationships and dynamics as well as potentially tell stories and provide grist for evaluation and hypotheses

Represented the multiple dimensions of value and different kinds of data, specifically focused on addressing a contributor as a valued member of the community or a vandal

Questions focused on visual comparisons of the same data or problem space to compare two different designs on the same issue

The original synthesis lead to a number of visualizations widely ranging in approach. Though the underlying function of each of the visualizations was to convey relationships, the graphics were either chart or scenic and were intended to not only convey understanding of the users but also personal biases. These scenic representations were influenced by the work of John Statsco.

The questions revolved around comprehension and preference of the visualizations presented. We also asked open-ended opinion questions to better understand these responses and community bias.

Graphical Test Bed

Word cloud representing frequency of interaction by users

The following are a number of the visual representations presented to the survey participants.

These first two representations are intended to portray the same information: amount of interaction between participants. Here, size and weight of the name represented a larger amount of interaction with an editor (who's name is not on this list). This word cloud was paired with...

Graphical Test Bed

Storytelling graphic representing frequency of interaction of users, including the awarding of markers of appreciation called barnstars.

a storytelling graphic of a beach scene.

Here, the closeness of a beach umbrella to the water represented closeness to the editor (also not represented) by amount of interaction.

Graphical Test Bed

Bar chart representing interaction of user through markers of appreciation called barnstars, thanks and reversion.

Respondents were then asked to analyze two graphics displaying the action of vandal-fighting and appreciated behavior, which is the removal of another editor's text because it was invalid, spam, unreferenced, or improperly unformatted. In the wants/needs analysis these were the two top activities that participant's want represented.

This representation is a very straightforward bar chart: reversions received and executed, as well as barnstars, which is a formalized token delivered from one editor to another on their user page. These tokens can be either positive or negative

Graphical Test Bed

Storytelling graphic representing interaction of user through markers of appreciation called barnstars, thanks and reversion.

The apples in this tree represent 150 reverts by the editor to other's work. The fallen apples represent 150 reverts to this editor's work. The flowers represent an expression of thanks (such as saying thank you on the user's user page). The butterflies represent that the editor has received 1 barnstar.

Results:
Self-Perception and Visual Effectiveness

Quantitative results demonstrated a dichotomy we refer to as preference vs performance

Qualitative results demonstrated self-perception and social constraints

We analyzed the quantitative survey elements to uncover patterns in the response and then used coding of the open-ended questions to attempt to color in the reasoning for the responses.

Surprising to our team, the most basic of analysis revealed quite interesting results!

Results:Preference vs Performance43 %

57 %

Number of respondents who preferred the graphic



Results:
Preference vs. Performance

preferred

49.12%

8.10%Can't Decide

preferred

42.10%



Preference vs. Performance

Of the respondents who determined the Tree elicited the most understanding only 50% interpreted the display correctly.



Describing the Numbers

The social aspect part of your chart isn't something I look at much now, mostly because it's not easy to identify.

You don't really care about the precise numbers - just the impression[The bar chart]'s simpler; the tree is just ... I don't know. Odd. :-)

We learned graphs in school since we were all little and the tree does nto look serious and very hard to under stand. The tree is offensive because it makes me feel like a pre-schooler learning how to say the alphabet (A is for apple, B is for butterfly)...

I'm a follower of Edward Tufte not a member of the Tufty Club

So what did we learn from this study and how are we moving forward?

Through a re-evaluation of the initial interviews and an analysis of the open-ended questions from the surveys, we have designated activities that are not only critical to the collaborative environment but are also of most interest to the editors as data for tools to support their work.

Through this we have generated a list of basic data types which have informed the objects in the object-oriented design of the Re:Flex system We have begun building these visualizations through these data types

Discussion:
Lessons from the Wikipedia community

Identified activities that are most valuable to the collaborative effort

Identified data types which constitute these activities (basis for both calculation and low level representation)

Identified biases of the collaborative community

Usage patterns and potential trajectories for system development

So what did we learn from this study and how are we moving forward?

Through a re-evaluation of the initial interviews and an analysis of the open-ended questions from the surveys, we have designated activities that are not only critical to the collaborative environment but are also of most interest to the editors as data for tools to support their work.

Through this we have generated a list of basic data types which have informed the objects in the object-oriented design of the Re:Flex system We have begun building these visualizations through these data types

Future Work

Develop neutral representations of interaction for visualizations

Iteration of survey to validate effectiveness of updated visualizations

Integrate into collaborative information visualization toolset, Re:Flex

We are currently in the process of integrating all of these components into our system. Unfortunately I do not have a demo to present at this talk, but if anyone is interested please contact me.

Using these visualizations, we will be performing an iteration of an updated survey to validate the effectiveness of the more neutrally (where we believe neutral exists for this community) represented information visualizations. This survey will add a component of interaction to these information visualizations, where different activity types represented can be altered and focal points can be rearranged.

Questions?

Stephanie [email protected]

Mark ZachryDavid McDonaldElly SearleBabbyCarol Allen

Travis KripleanIvan BestachnickAlena BensonToni FerroKatie Derthick

Special Thanks to:

NSF Grant #NSF IIS-0811210

Click to edit the title text formatClick to edit Master title style

4/11/11

Click to edit the title text formatClick to edit Master title style

Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelEighth Outline Level

Ninth Outline LevelClick to edit Master text styles

Second level

Third level

Fourth level

Fifth level

4/11/11

Tree Scene Interaction PerceptionColumn I

Can't Tell/Incorrect Response50

Correct Response50

Beach Scene Interaction PerceptionColumn F

Highly Interactive14

Somewhat Interactive35

Can't tell by looking at the image8

Of the people who preferred each representation type, how did they perform?AccurateInaccurate

Graph0.64910

Tree0.070150.07015

Cloud0.0240.0181

Beach0.41120.0399

Word Cloud Interaction PerceptionColumn F

Highly Interactive15

Somewhat Interactive18

Can't tell by looking at the image25

designing for collaboration: challenges & considerations of multi-use information visualization...

Technology