designing for collaboration: challenges & considerations of multi-use information visualization...
Post on 16-Apr-2017
775 Views
Preview:
TRANSCRIPT
Designing for Collaboration
Challenges and Considerations of Multi-Use Information
Visualization Tools
Stephanie Gokhman
April 8, 2011Allen Library
Introduction:
I'm Stephanie Gokhman first year PhD student jointly affiliated
with the Communicative Practices Lab with Mark Zachry and the
Computer-Supported Collaboration Lab with Charlotte Lee. The
research presented here is a small piece of a multi-year NSF-funded
project with Mark and David McDonald from the iSchool.
Overview
Introduction: What is Re:Flex? Why Wikipedia?
Literature Review: What do we know about building systems to support collaboration through visualizations?
Methods: How do we expose the considerations in designing information visualization tools for collaboration?
Results & Discussion: What did we find and how does this give us insight into collaborative information visualization tools?
Future Work
This research is just one of many stages and offshoots of that funded initiative. We are developing an information visualization toolset for use by Wikipedians and for use by us to better understand the collaborative effort that creates Wikipedia. We call this Re:Flex, which stands for Reflexive Flexible Reputation tool. It also supports a number of other verbs with RE and FLEX (such as REminisce, REtrospection, etc). We'll briefly discuss the greater overall Re:Flex project and why we selected Wikipedia later, but this particular talk is focused on foundational work to inform the information visualizations that comprise the toolset...
This research is just one of many stages and offshoots of that funded initiative. We are developing an information visualization toolset for use by Wikipedians and for use by us to better understand the collaborative effort that creates Wikipedia. We call this Re:Flex, which stands for Reflexive Flexible Reputation tool. It also supports a number of other verbs with RE and FLEX (such as REminisce, REtrospection, etc). We'll briefly discuss the greater overall Re:Flex project and why we selected Wikipedia later, but this particular talk is focused on foundational work to inform the information visualizations that comprise the toolset...
This research is just one of many stages and offshoots of that funded initiative. We are developing an information visualization toolset for use by Wikipedians and for use by us to better understand the collaborative effort that creates Wikipedia. We call this Re:Flex, which stands for Reflexive Flexible Reputation tool. It also supports a number of other verbs with RE and FLEX (such as REminisce, REtrospection, etc). We'll briefly discuss the greater overall Re:Flex project and why we selected Wikipedia later, but this particular talk is focused on foundational work to inform the information visualizations that comprise the toolset...
This research is just one of many stages and offshoots of that funded initiative. We are developing an information visualization toolset for use by Wikipedians and for use by us to better understand the collaborative effort that creates Wikipedia. We call this Re:Flex, which stands for Reflexive Flexible Reputation tool. It also supports a number of other verbs with RE and FLEX (such as REminisce, REtrospection, etc). We'll briefly discuss the greater overall Re:Flex project and why we selected Wikipedia later, but this particular talk is focused on foundational work to inform the information visualizations that comprise the toolset...
Information Visualizations to Support Collaboration
What are the steps to appropriately design information visualization tools to support collaboration?
The need for these tools is apparent but the method of designing them is not. Therefore this research was intended to answer the question What are the steps to appropriately design information visualization tools to support collaboration?
Re:Flex:
The Big Picture
Communicative Practices in Virtual Workspaces Lab
Set of tools to expose and visualize the relationships between editors and editors and editors and articles on Wikipedia
Focused on the types of work present in the Wikipedia community
Help Wikipedians recognize and value work in the Wikipedia community
Re:Flex is a reflexive, flexible reputation toolset using information visualizations to expose relationships between editors and other editors as well as editors and articles, or namespaces
Through these visualizations we hope to understand how to build systems to support collaborative spaces by exposing and visualizing relationship data. This is referred to as social translucence.
Why Wikipedia?
Largest freely accessible online collaboration
Largest, most popular reference work on the Internet, according to Alexa Statistics1
Characteristics of other collaborative spaces which these visualizations displayRelationshipsEdit historiesConflictUser pagesNegotiation
Multiplicity of potential spaces where information visualization would serve as meaningful tools
We selected Wikipedia to begin this research because it is the largest online collaboration, with all data since it's inception freely accessible, and contains a multiplicity of collaborative activities that are common across multiple contributor systems.
Say the sentences put a picture of the homepage
The BIGGER Picture:
Beyond Wikipedia
Allow large data sets, specifically edit histories, to be explored by many contributors collectively and individually
view data in context
observe patterns
make comparisons through visualizations
further understanding the community that has generated the body of knowledge
The visualizations are being designed with broader applicability to contributor systems in mind. These are systems in which public engagement provides content, where there exist user pages and edit histories as markers of contribution.
When building Re:Flex we determined components of collaboration in these types of systems that we viewed as most meaningful and best supported by information visualizations: viewing data in context, observing patterns, making comparisons through visualizations by synthesizing the large data sets and generally better understanding the community that has generated the body of knowledge.
Beyond Wikipedia
Underlying principles of Re:Flex serve a basic need across a number of collaborative technologies
Could be applied to other popular contributor systems: Quora1
GitHub2
TED Conversations3
1. http://www.quora.com2. https://www.github.com3. http://www.ted.com/conversations
These are a series of examples of contributor systems which also provide a context in which support for those activities would be meaningful.
Quora is a collection of publicly created questions and answers, sometimes referred to as a knowledge market for it's available knowledge resources.
GitHub is a platform that hosts open source coding projects where public participants edit and add to applications.
TED Conversations is platform similar to Quora, yet is not restricted to questions an answers. This platform mimics what some of us would see as a messageboard style platform and dynamic.
Information Visualizations
to Support Collaboration
As collaborative technologies increase in attention and prevalence, there exists a blossoming design space for information visualization tools built for collaborative environments
The complexity of collaborative spaces reveals a large number of tensions within this design space that have not been fully exploredWhat are the steps to appropriately design information visualization tools for collaboration?
This research is just one of many stages and offshoots of that funded initiative. We are developing an information visualization toolset for use by Wikipedians and for use by us to better understand the collaborative effort that creates Wikipedia. We call this Re:Flex, which stands for Reflexive Flexible Reputation tool. It also supports a number of other verbs with RE and FLEX (such as REminisce, REtrospection, etc). We'll briefly discuss the greater overall Re:Flex project and why we selected Wikipedia later, but this particular talk is focused on foundational work to inform the information visualizations that comprise the toolset...
Informing the Design of Re:Flex
What are the salient activity types?
What are the fundamental data types?
What are the biases of the users of Wikipedia?
What relationships do the users want to see visualized?
What relationships do the users need to see visualized?
Methodology
Interview, Wants/Needs analysisInformed visualizations built by a graphic designer from the Information School
Surveyed 100+ Wikipedians
Visual comparisons of the same data or problem space to compare two different designs on the same issue
Represented the multiple dimensions of value and different kinds of data, specifically focused on addressing a contributor as a valued member of the community or a vandal
To answer this question, we employed a jointly qualitative and quantitative approach to better understand the types of data the community values, the types of visualizations the community finds preferable, and inherent social biases and constraints.This data collection began prior to my involvement in the project by Elly Searle. We invited Wikipedians from the Puget Sound region (from Portland to Vancouver, BC) to deeply understand their individual usage and community behaviors.By conducting a wants/needs analysis on a series of interviews with active Wikipedians she selected a number of primary interaction/activity types and fundamental data types which inform these activities. Using this information we hired a graphic designer from the User-Centered Design program to create a series of visualizations which we then tested through a survey of over 100 active Wikipedians.
Synthesis of Interviews:
Uncovering Salient Data and Action Types
Actions:Content creation
Vandal-fighting
Quality control
Reversion
Appreciation
Data Types:Editor
Total Number of edits by editor
Articles
Barnstars
Namespaces
The above list is the result of a second pass at the interview materials from the wants/needs analysis. These are the activity types and fundamental data points that we are using to inform the current iteration of visualization design.
Graphical Test Bed
Visual metaphor to convey relationships and dynamics as well as potentially tell stories and provide grist for evaluation and hypotheses
Represented the multiple dimensions of value and different kinds of data, specifically focused on addressing a contributor as a valued member of the community or a vandal
Questions focused on visual comparisons of the same data or problem space to compare two different designs on the same issue
The original synthesis lead to a number of visualizations widely ranging in approach. Though the underlying function of each of the visualizations was to convey relationships, the graphics were either chart or scenic and were intended to not only convey understanding of the users but also personal biases. These scenic representations were influenced by the work of John Statsco.
The questions revolved around comprehension and preference of the visualizations presented. We also asked open-ended opinion questions to better understand these responses and community bias.
Graphical Test Bed
Word cloud representing frequency of interaction by users
The following are a number of the visual representations presented to the survey participants.
These first two representations are intended to portray the same information: amount of interaction between participants. Here, size and weight of the name represented a larger amount of interaction with an editor (who's name is not on this list). This word cloud was paired with...
Graphical Test Bed
Storytelling graphic representing frequency of interaction of users, including the awarding of markers of appreciation called barnstars.
a storytelling graphic of a beach scene.
Here, the closeness of a beach umbrella to the water represented closeness to the editor (also not represented) by amount of interaction.
Graphical Test Bed
Bar chart representing interaction of user through markers of appreciation called barnstars, thanks and reversion.
Respondents were then asked to analyze two graphics displaying the action of vandal-fighting and appreciated behavior, which is the removal of another editor's text because it was invalid, spam, unreferenced, or improperly unformatted. In the wants/needs analysis these were the two top activities that participant's want represented.
This representation is a very straightforward bar chart: reversions received and executed, as well as barnstars, which is a formalized token delivered from one editor to another on their user page. These tokens can be either positive or negative
Graphical Test Bed
Storytelling graphic representing interaction of user through markers of appreciation called barnstars, thanks and reversion.
The apples in this tree represent 150 reverts by the editor to other's work. The fallen apples represent 150 reverts to this editor's work. The flowers represent an expression of thanks (such as saying thank you on the user's user page). The butterflies represent that the editor has received 1 barnstar.
Results:
Self-Perception and Visual Effectiveness
Quantitative results demonstrated a dichotomy we refer to as preference vs performance
Qualitative results demonstrated self-perception and social constraints
We analyzed the quantitative survey elements to uncover patterns in the response and then used coding of the open-ended questions to attempt to color in the reasoning for the responses.
Surprising to our team, the most basic of analysis revealed quite interesting results!
Results:Preference vs Performance43 %
57 %
Number of respondents who preferred the graphic
We analyzed the quantitative survey elements to uncover patterns in the response and then used coding of the open-ended questions to attempt to color in the reasoning for the responses.
Surprising to our team, the most basic of analysis revealed quite interesting results!
Results:
Preference vs. Performance
preferred
49.12%
8.10%Can't Decide
preferred
42.10%
We analyzed the quantitative survey elements to uncover patterns in the response and then used coding of the open-ended questions to attempt to color in the reasoning for the responses.
Surprising to our team, the most basic of analysis revealed quite interesting results!
Preference vs. Performance
Of the respondents who determined the Tree elicited the most understanding only 50% interpreted the display correctly.
We analyzed the quantitative survey elements to uncover patterns in the response and then used coding of the open-ended questions to attempt to color in the reasoning for the responses.
Surprising to our team, the most basic of analysis revealed quite interesting results!
Describing the Numbers
The social aspect part of your chart isn't something I look at much now, mostly because it's not easy to identify.
You don't really care about the precise numbers - just the impression[The bar chart]'s simpler; the tree is just ... I don't know. Odd. :-)
We learned graphs in school since we were all little and the tree does nto look serious and very hard to under stand. The tree is offensive because it makes me feel like a pre-schooler learning how to say the alphabet (A is for apple, B is for butterfly)...
I'm a follower of Edward Tufte not a member of the Tufty Club
So what did we learn from this study and how are we moving forward?
Through a re-evaluation of the initial interviews and an analysis of the open-ended questions from the surveys, we have designated activities that are not only critical to the collaborative environment but are also of most interest to the editors as data for tools to support their work.
Through this we have generated a list of basic data types which have informed the objects in the object-oriented design of the Re:Flex system We have begun building these visualizations through these data types
Discussion:
Lessons from the Wikipedia community
Identified activities that are most valuable to the collaborative effort
Identified data types which constitute these activities (basis for both calculation and low level representation)
Identified biases of the collaborative community
Usage patterns and potential trajectories for system development
So what did we learn from this study and how are we moving forward?
Through a re-evaluation of the initial interviews and an analysis of the open-ended questions from the surveys, we have designated activities that are not only critical to the collaborative environment but are also of most interest to the editors as data for tools to support their work.
Through this we have generated a list of basic data types which have informed the objects in the object-oriented design of the Re:Flex system We have begun building these visualizations through these data types
Future Work
Develop neutral representations of interaction for visualizations
Iteration of survey to validate effectiveness of updated visualizations
Integrate into collaborative information visualization toolset, Re:Flex
We are currently in the process of integrating all of these components into our system. Unfortunately I do not have a demo to present at this talk, but if anyone is interested please contact me.
Using these visualizations, we will be performing an iteration of an updated survey to validate the effectiveness of the more neutrally (where we believe neutral exists for this community) represented information visualizations. This survey will add a component of interaction to these information visualizations, where different activity types represented can be altered and focal points can be rearranged.
Questions?
Stephanie Gokhmansgokhman@uw.edu
Mark ZachryDavid McDonaldElly SearleBabbyCarol Allen
Travis KripleanIvan BestachnickAlena BensonToni FerroKatie Derthick
Special Thanks to:
NSF Grant #NSF IIS-0811210
Click to edit the title text formatClick to edit Master title style
4/11/11
Click to edit the title text formatClick to edit Master title style
Click to edit the outline text formatSecond Outline LevelThird Outline LevelFourth Outline LevelFifth Outline LevelSixth Outline LevelSeventh Outline LevelEighth Outline Level
Ninth Outline LevelClick to edit Master text styles
Second level
Third level
Fourth level
Fifth level
4/11/11
Tree Scene Interaction PerceptionColumn I
Can't Tell/Incorrect Response50
Correct Response50
Beach Scene Interaction PerceptionColumn F
Highly Interactive14
Somewhat Interactive35
Can't tell by looking at the image8
Of the people who preferred each representation type, how did they perform?AccurateInaccurate
Graph0.64910
Tree0.070150.07015
Cloud0.0240.0181
Beach0.41120.0399
Word Cloud Interaction PerceptionColumn F
Highly Interactive15
Somewhat Interactive18
Can't tell by looking at the image25
top related