Building Systems to Improve OnlineDiscussion

Amy X. ZhangMIT CSAILCambridge, MAaxz@mit.edu

Permission to make digital or hard copies of part or all of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page. Copyrights for third-party components of this work must be honored.For all other uses, contact the owner/author(s).

Copyright held by the owner/author(s).CSCW ’18 Companion, November 3–7, 2018, Jersey City, NJ, USAACM 978-1-4503-6018-0/18/11.https://doi.org/10.1145/3272973.3272975

AbstractLittle has changed in the design of online discussion sys-tems in the decades they have been available, even asproblems involving scale, loss of context, and bad actorsmount with broader use. To solve these problems, my re-search is on building novel online discussion systems thatgive users direct control over their experiences and informa-tion. Specifically, I focus on: 1) summarization tools to makesense of large discussions, 2) annotation tools to situateconversations in the context of what is being discussed, aswell as 3) moderation tools to give users more fine-grainedcontrol over governance and delivery of messages.

Author Keywordsonline discussion; forums; chat; social media; email; sum-marization; annotation; tagging; sensemaking; deliberation;moderation; online communities

IntroductionDiscussions on systems such as email, forums, and chathave been pervasive on the internet since its inception.They contain a diversity of rich information and experi-ences, including differing opinions, anecdotes, humor, ex-planations, coordination, and deliberation [1]. However, on-line discussion tools are still remarkably primitive, barelychanged from their origins in forums, email, and instantmessaging. As a result, problems with discussion systems

Doctoral Consortium CSCW’18 Companion, November 3–7, 2018, Jersey City, NJ, USA

65

persist and are now exacerbated by growing participation inonline discussion and new discussion domains.

Figure 1: Wikum: a tool forsummarizing long discussions.

Figure 2: Tilda: a tool to mark upand summarize group chat.

Figure 3: Eyebrowse: bringingsocial experiences to browsing.

Too often, discussions online lead to polarizing interactionsor peter out with no resolution, wasting valuable time andeffort between participants. Sometimes this happens be-cause of the scale of discussion, making it difficult for par-ticipants to get an overview of what happened or refer back.A related problem is the loss of context when discussionsare stripped of information about their particular time, place,origination topic, or make-up of participants. Other timesthe problems are due to bad actors or contested norms,leading to tensions between participants and, in the worstcases, escalating into harassment of individuals.

In my thesis, I introduce a diverse set of systems that reimag-ine outdated discussion designs to mitigate the problemsthat discussion systems face. These systems confront thegrowing pains of discussion systems by providing tools tousers and communities that give them greater controlover their experiences and information online.

Proposed Dissertation WorkMy work can be divided into three main areas of inquiry thateach examine design considerations around a particulartechnique or lever of control.

1) Summarization Tools to Make Sense of Large DiscussionsThe first set of systems help make sense of large discus-sions by providing collective summarization capabilitieswithin discussion systems to provide a mechanism for peo-ple to deal with scale as well as a way to reflect on whatwas discussed. Because summaries are time-consumingand difficult to create, the project Wikum1 [9] considershow a group of people could individually contribute small

1wikum.org

amounts of work to refine a large discussion into a dynamictextual summary, called a summary tree that can be ex-plored at varying levels of detail. We design a workflowfor creating a summary tree using recursive summariza-tion, where users build summaries of small sections of thediscussion, small sets of those summaries are then aggre-gated and summarized, and so on, until the entire discus-sion is summarized. We are next researching Wikum’s po-tential as a tool for Wikipedia editors to summarize discus-sions on Talk Pages [2]. A second tool Tilda2 [6] focuses onsummarization for group chat. The tool works by allowingparticipants to mark up an ongoing continuous chat streamto turn it into discrete structured summaries. This allowsusers to easily get an overview of the chat for when they fallbehind. From interviews, we discovered what informationusers want to glean from chat as well as alternative designsfor chat summaries, and then developed and deployed Tildato several active Slack groups for work.

In relation to this work, I am interested in continuing to ex-plore how humans and automatic techniques can work intandem to construct summaries. My work on the charac-terization of different types of discussion using commondiscourse sequences [7] is one step towards understand-ing how summarization needs and workflows for creatingsummaries change depending on the type of conversation.

2) Annotation Tools to Situate Conversations in ContextThe second set of systems situate conversations in theproper context by providing annotation capabilities whenhaving discussions. One way to provide context with an-notation is to allow discussions to live as annotations inthe “margins" of other primary content, such as webpages,articles, or textbooks. An example is Eyebrowse3 [5], a

2tildachat.com3eyebrowse.csail.mit.edu

Doctoral Consortium CSCW’18 Companion, November 3–7, 2018, Jersey City, NJ, USA

66

browser extension and web tool that allows users to shareaspects of their browsing history, such as when they areactive on particular sites. This allows people to “bump intoeach other" in spaces on the web and have serendipitouslive conversations or asynchronous discussions that arethen tied to the pages they are on. Another way to pro-vide context with annotation is to annotate the commentsthemselves with relevant information. Here, I focus on self-annotation by commenters to both reveal the underlying ba-sis of their comment and encourage self-reflection. For in-stance, Pano4 is a tool that allows people to tag commentsand articles with moral framing in order to become awareof their own and others’ underlying moral values. A 10-dayfield study we conducted provided evidence that users im-proved their ability to frame arguments in the values of theopposing side.

Figure 4: Pano: annotation ofcomments with moral framing.

Figure 5: NB: adding emojis tocomments to capture emotionalreactions to textbook passages.

Finally, systems like Nota Bene (NB)5 [10] demonstratethat the combination of both types of annotation can beuseful. NB allows students to have discussions in the “mar-gins" of online textbooks anchored to a specific passageon the page. This allows students to read and concurrentlyview discussions in context of what is being discussed. Inmy work related to NB, I consider how self-annotation ofemotional state within a comment using emojis and hash-tags could provide signals for readers about what parts of atextbook to focus on [8]. This could also help instructors cutthrough the noise of discussion to find unanswered ques-tions or resolve disputes, or distinguish interesting fromconfusing passages for future revision.

3) Moderation Tools for Finer-Grained Control of DeliveryFinally, the third set of systems provide more powerful mod-eration tools that give users finer-grained control over the

4pano.csail.mit.edu5nb.mit.edu

delivery of content, including what messages they see, whoin turn sees their messages, as well as how messages getaccessed and delivered. This can help manage informationoverload as well as help a community deal with contestednorms. From studying existing communities, we notice var-ied perspectives as to how members of the same mailinglist should access and post to the list, leading to tensions,as well as a general hesitancy towards posting [4].

One solution is to give users more fine-grained control overdelivery of content, so that all members of a community areworking together to ensure content gets delivered only tothose who want to receive it. Drawing from our researchinto tensions within online groups, we built Murmur6, amailing list system that aims to keep the benefits of email,such as greater confidence that messages will be seen,while introducing new features that are present in moremodern systems such as Facebook, such as social mod-eration. Rather than using algorithmic curation, which putsthe delivery of content in the hands of a model, Murmur al-lows users to have more explicit and fine-grained control tofilter, block, follow, and otherwise curate how and to whomdiscussions are sent and received.

We also study how novel moderation capabilities can com-bat problems such as harassment. We developed the toolSquadbox7, where people facing harassment in their per-sonal inboxes can recruit their friends as moderators [3].From interviews with 18 recipients of online harassment,we learned how users wish to personalize their strategy fordealing with what they consider harassment, and designedSquadbox to be customizable to their needs.

6murmur.csail.mit.edu7squadbox.org

Doctoral Consortium CSCW’18 Companion, November 3–7, 2018, Jersey City, NJ, USA

67

Expected ContributionsIn my dissertation research, I hope to make the followingcontributions: 1) learn from specific communities of peopleengaging in online discussion to understand their needs,problems, and existing strategies; 2) explore the designspace of possible user-led and community-led controls toenrich, organize, and govern discussion; and 3) producedesign recommendations for social media platforms andcommunities to support better conversations at scale.

Figure 6: Murmur: configurablemailing list system.

Figure 7: Squadbox: combatingharassment using friendsourcedmoderation.

Goals for CSCW Doctoral ColloquiumOne of my goals is to gather feedback on the direction andframing of my thesis topic as it stands. In my Ph.D. thus far,I have worked on a number of different projects and am nowreflecting on both the coherence of my work and whetherthere are significant gaps in what I propose to cover. In thefinal year of my program, I am interested in suggestions onnew work or extensions of existing work I should embarkupon to narrow down or round out my current direction ofinquiry. A second goal is to broaden my horizons to makeinteresting connections to academic disciplines, communi-ties, and research problems that are adjacent to my work.I would like to improve how my work points to and sits incontext to historical and existing lines of research.

REFERENCES1. Justin Cranshaw and Aniket Kittur. 2011. The polymath

project: lessons from a successful online collaborationin mathematics. In Proceedings of the SIGCHIConference on Human Factors in Computing Systems.ACM, 1865–1874.

2. Jane Im, Amy X. Zhang, Christopher J. Schilling, andDavid Karger. 2018. Deliberation and Resolution onWikipedia: A Case Study of Requests for Comments.Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article74 (Nov. 2018), 24 pages.

3. Kaitlin Mahar, Amy X. Zhang, and David Karger. 2018.Squadbox: A Tool to Combat Email Harassment UsingFriendsourced Moderation. In Proceedings of the 2018CHI Conference on Human Factors in ComputingSystems (CHI ’18). ACM, New York, NY, USA, Article586, 13 pages.

4. Amy X. Zhang, Mark S. Ackerman, and David R.Karger. 2015. Mailing Lists: Why Are They Still Here,What’s Wrong With Them, and How Can We FixThem?. In CHI. ACM, New York, NY, USA, 4009–4018.

5. Amy X. Zhang, Joshua Blum, and David Karger. 2016.Opportunities and Challenges Around a Tool for Socialand Public Web Activity Tracking. In CSCW. ACM.

6. Amy X. Zhang and Justin Cranshaw. 2018. MakingSense of Group Chat through Collaborative Taggingand Summarization. Proc. ACM Hum.-Comput.Interact. 2, CSCW, Article 196 (Nov. 2018), 27 pages.

7. Amy X. Zhang, Bryan Culbertson, and PraveenParitosh. 2017a. Characterizing Online DiscussionUsing Coarse Discourse Sequences. In ICWSM. AAAI.

8. Amy X. Zhang, Michele Igo, Marc Facciotti, and DavidKarger. 2017b. Using Student Annotated Hashtags andEmojis to Collect Nuanced Affective States. In Learning@ Scale. ACM.

9. Amy X. Zhang, Lea Verou, and David Karger. 2017c.Wikum: Bridging Discussion Forums and Wikis UsingRecursive Summarization. In CSCW. ACM.

10. Sacha Zyto, David Karger, Mark Ackerman, and SanjoyMahajan. 2012. Successful classroom deployment of asocial document annotation system. In CHI. ACM,1883–1892.

Doctoral Consortium CSCW’18 Companion, November 3–7, 2018, Jersey City, NJ, USA

68