Crowdsourcing Knowledge: Interactive Learning with Mapping Historic Skies Samantha Blickhan (sblickhan@adlerplanetarium.org ) The Adler Planetarium, Chicago IL; Jessica BrodeFrank ( jbrodefrank@adlerplanetarium.org ) The Adler Planetarium, Chicago IL; Becky Rother (becky@zooniverse.org ) The Adler Planetarium, Chicago IL. Abstract This paper will describe the process of creating Mapping Historic Skies, a project which aims to crowdsource the identification of constellations in 4,000 digital images from the Adler Planetarium’s collection of historical star maps. The project is a collaboration between members of the Adler Collections department and Zooniverse team members from the Adler’s department of Citizen Science. We describe the process of developing, prototyping and testing the project using the Zooniverse Project Builder ( https://www.zooniverse.org/lab ) and mobile app, as well as the challenges of designing project workflows to be part of an in-museum exhibit, as compared with workflows hosted on an online platform. The project will ultimately be included on an interactive touchscreen in the Adler’s upcoming Chicago’s Night Sky exhibit, opening November 2019. Members of the Mapping Historic Skies project team will be soliciting feedback from MuseWeb attendees during the app crit portion of MW19. Keywords : citizen science, art history, astronomy, research, crowdsourcing, outreach, exhibits Introduction The Adler Planetarium is a unique institution, in that it not only serves the city of Chicago, IL, but a worldwide community. This international reach is thanks in large part to our involvement with the Zooniverse platform ( https://www.zooniverse.org ). A partnership between the Adler and Oxford University, Zooniverse is the world-leading platform for online crowdsourced research. Zooniverse began in 2007 as a single project, Galaxy Zoo (Lintott et al., 2008), but now comprises over 80 currently-active projects and more than 1.7 million registered volunteers in 234 countries around the world. Zooniverse facilitates crowdsourced research by connecting everyday citizens with researchers to help classify galaxies, find new planets, transcribe handwritten historical documents, and complete dozens of original projects in astronomy, biology, science, and the humanities. To mitigate against inaccurate data, Zooniverse projects feature blind classification. To participate in a project, volunteers engage in a task or series of tasks independently of one another, and the responses are aggregated together for consensus. The resulting data from these projects have been used in more than 150 peer-reviewed publications to date (https://www.zooniverse.org/about/publications ). In 2015, the Zooniverse launched the Project Builder ( https://www.zooniverse.org/lab ), a do-it-yourself tool that allows anyone to build a crowdsourcing project on the Zooniverse platform for free (Trouille et al., 2017).

As the Zooniverse has grown, so have the Adler’s hopes of using the platform to shine a light on in-house collections of books, maps, and objects, and invite members of the public to engage with the curatorial work being done. One such project idea came out of a multi-year digitization project funded by the National Endowment for the Humanities (Grant #PW-51687-14). Collections team members at the Adler were able to digitize the institution’s collection of over 4,000 historical constellation maps and depictions covering six hundred years, featuring artistic representations of the sky from around the world. The initial concept was to use these images to show how the constellations we know today have been depicted in art over time and throughout various cultures. The biggest challenge to this goal was the sheer size of the dataset. With only one curator and one full time digital collections staff member, an undertaking of this size would have spanned years. Realizing that crowdsourcing might be a viable option for such an undertaking, the Collections team reached out to the Zooniverse team, based in the Adler’s Citizen Science department, for help. The end result is a single, Zooniverse-hosted project that combines a traditional, online crowdsourcing project with a mobile app and an in-gallery interactive adapted from the mobile app. History of the Project In 2014, the Adler Collections team proposed an early version of the current project to the NEH for funding. It was called Digital Historic Skies and the goal was to create an interactive mobile application that would teach the general public about art, history, and science in cultures throughout the world through the comparison of historical celestial maps and images of the current night sky. The ultimate goal was to create an app for smartphones that would use GPS to pull associated images from the Adler’s celestial cartography collection, allowing users to look at any region of the sky and easily access the relevant historical and cultural constellation depictions. The Digital Historic Skies proposal received funding from the NEH for prototyping research (Grant #HD-51957-14), and resulted in a white paper (Raposo 2016) which offered suggestions for continued development of the project and set in motion the actions which ultimately formed the foundation for the current effort. Another obstacle for the project was the small team size. Early planning for Digital Historic Skies was stalled due to the need for restructuring in order to make the goals achievable for such a small team. For example, the original proposal did not foresee the amount of time needed to comb through the celestial cartography collection. Not only did each image need to be catalogued, but every individual constellation within each image needed to be identified, cropped out of the original digital image, and formatted for this specific use within the app. Though there is still hope to have an end product of the type imagined in the original Digital Historic Skies proposal, it is no longer listed as an immediate outcome. The focus is now on smaller-scale, achievable goals, detailed below, which can then be used to support long-term project development. Through conversations between the Collections department and the Zooniverse team at the Adler, the newest version of the project emerged, entitled Mapping Historic Skies. The project consists of an interactive crowdsourcing project built using the Zooniverse Project Builder. The

end goal is to narrow down the large Adler dataset into images of single, identified constellations. The interactive workflows on Zooniverse break down the identification of constellations in the Adler’s historical star maps into simple tasks achievable by anyone without requiring previous knowledge of astronomy. The intended audience for this interactive is visitors to the Adler’s upcoming Chicago’s Night Sky exhibit, as well as online participants from all over the world, although these two groups of volunteers will have different options of tasks in which they can participate. In time, the resulting identifications will be added to a database of constellations, meaning online volunteers as well as visitors to the exhibit who participate in Mapping Historic Skies are helping with real research and are participating as co-creators towards a repository of public knowledge. The creation of the database will be part of the next project phase. Interactive Workflows in Mapping Historic Skies Before starting the design and development process, the Collections and Zooniverse teams worked with the Adler’s Guest Experience team to create internal project objectives to direct the development of the in-museum interactive. Mapping Historic Skies Internal Project Objectives:

1. Connect within your institution to the people best suited to each role. Cross-departmental collaboration is key in innovative digital projects.

2. Assess results of your trial runs and utilize advice to advance your prototype. Test it, test it, test it, then workshop and try again.

3. Visitor participation in trials will lead to a better experience for guests on the final format. Interactive citizen science allows visitors to take ownership of the collection in a very real and interactive way.

The project is divided into three workflows: image sorting, image segmentation, and constellation identification. Many examples in the Adler collection feature multiple constellations within a single image. For example, Figure 1 below, from Rev. Richard Rouse Bloxam’s Urania's mirror: or, A view of the heavens (engraved by Sidney Hall in 1825) , includes four different constellations: Sagittarius (the centaur aiming a bow), Corona Australis, the Southern Crown (the wreath), Microscopium and Telescopium (the microscope and telescope, respectively).

Figure 1: Excerpt from Sidney Hall, Urania's mirror: or, A view of the heavens , plate 24. Published by Samuel Leigh, 1825. To facilitate the identification of individual constellations within an example like the one above, the multi-constellation images first need to be identified and separated from the full dataset. This workflow will be available on Zooniverse.org as well as the Zooniverse mobile app (available at https://itunes.apple.com/us/app/zooniverse/id1194130243?mt=8 and https://play.google.com/store/apps/details?id=com.zooniversemobile ). Volunteers will be shown an image from the Adler collection and asked if the image depicts more than 4 constellations with the option to respond ‘Yes’ or ‘No’ (justification for the choice of 4 constellations is discussed below). Multiple volunteers will answer this question for each image, and the answers will be aggregated together for consensus. Figure 2 shows what this workflow will look like in the Zooniverse iPad app.

Figure 2: Zooniverse iPad app. The images that receive a ‘No’ consensus (meaning there are 4 constellations or fewer in the image) will then be added to the segmentation workflow. This task will be presented to visitors in the Chicago’s Night Sky exhibit. The in-exhibit interactive workflow will be presented on a touchscreen, using a slightly modified version of the Zooniverse mobile app, shown in Figure 3. Modifications to the main Zooniverse.org app include the inability to navigate away from the Mapping Historic Skies project and a modified tutorial. While the Zooniverse codebase is open source and freely available on GitHub ( https://github.com/zooniverse ), customization for the ‘in-gallery’ version of the mobile app was available to the project team due to the Adler’s proximity to Zooniverse developers.

Figure 3: Zooniverse iPad app for museum. In this workflow, participants are invited to draw boxes around individual constellations within an image (or ‘subject’, in Zooniverse terms). As was the case with the Yes/No workflow, the boxes will be aggregated together and the coordinate data will be used to crop the multi-constellation subjects into separate images, so that each contains a single constellation. Once the multi-constellation images are cropped into individual images, the identification of individual constellations will be done by online volunteers on Zooniverse.org. Volunteers will be shown an image of a single constellation and asked if they know the name of the constellation depicted in the image. If the volunteers already know the answer, they can type the name of the constellation in manually and submit their response. If not, they are then invited to answer a series of questions based on the visual characteristics of the artistic depiction of the constellation. These questions are meant to guide volunteers through the process of identifying the constellation. The initial questions are quite general (“Choose the option that best represents the figure in the image: Human, Animal, or Object”), but narrow in specificity as the volunteer answers questions, finally resulting in either a recommendation of a single constellation name, or a choice between two or three similar constellations (“A ram with a tail: Capricorn” or “A ram without a tail: Aries”).

The decision to separate the three workflows between different audiences for the project was based on available technical infrastructure as well as a desire to provide a positive experience for exhibit guests and online visitors. The constellation identification workflow is only hosted on Zooniverse.org, because the mobile app does not yet have the functionality to ask multiple questions about a single subject. The Yes/No workflow will be available on both Zooniverse.org and the mobile app, but not in the museum even though the technical resources exist to host it there. This is because we wanted to keep the in-museum interactive to a single task, and the drawing task offers greater opportunities for guests to engage meaningfully with the constellations within the images. During testing, guests told us that they enjoyed the Yes/No swipe workflow, but we noticed that when participating in that task guests had a tendency to go into Auto-Pilot mode and swipe as quickly as possible, whereas with the drawing task they needed to spend more time looking at the image and use precision when drawing bounding boxes around the constellations. That information led us to choose the segmentation workflow (drawing boxes) as the option that would be shown to museum visitors. After going through all the Mapping Historic Skies workflows, the image of Sagittarius from Figure 1 would ultimately become four separate images, each classified as an individual constellation, shown in Figure 4.

Figure 4: ‘Classified’ constellations from Figure 1. Transferring Online Citizen Science into Museum Exhibits Though the Zooniverse team had previously worked on many similar image segmentation and identification projects, this one would be profoundly different. Instead of taking place solely on the Zooniverse website or mobile app, Mapping Historic Skies would include volunteer participation from within a gallery setting; instead of participants who visit the Zooniverse platform with the intent to participate in real, crowdsourced research, the in-museum iteration of

the project would need to welcome visitors and pique their curiosity while also communicating that this is a real research project and not an exploratory interactive. During the design of this process, our team looked at the MicroPlants project (http://microplants.fieldmuseum.org/), which similarly features an online project (with tools designed by the Zooniverse team) as well as an interactive kiosk hosted at the Field Museum in Chicago. In their findings, the MicroPlants team reported that ultimately the data provided by both online participants and kiosk visitors was comparable with data provided by experts, but noted “participant interaction with the kiosk is very different from their interaction with the online platform” and “[i]n the exhibit, participant interaction is potentially limited by time, interest, and other waiting people” (von Konrat et al. 2018). The MicroPlants team at the Field Museum is not the first to integrate online crowdsourcing into their exhibitions; the online crowdsourcing hub SciStarter offers a ‘kiosk’ for museums and other public spaces which allows institutional visitors to participate in online citizen science projects by choosing from a curated list (Cavalier 2012). However, the MicroPlants example was particularly relevant to the design of Mapping Historic Skies, as it featured a single project which received classifications from museum visitors as well as online volunteers, and the team was able to study the results received from both platforms. The Mapping Historic Skies project does not present the same task to online and in-museum volunteers, but it was still useful to look at the research produced by the MicroPlants team when designing for both these experiences, particularly in regard to data quality and participant interaction. As the Adler Collections team prepared to load the content into the Zooniverse platform (resizing images and identifying which objects would work well), the Zooniverse team created a beta version of the project workflow, asking volunteers whether an image depicted a single constellation (if so, swipe left), or multiple constellations (swipe right). At an after-hours event for members of the Adler Planetarium (Members’ Night), we took the beta prototype onto the floor and tested it with guests to judge if their reactions and enthusiasm for the task would match that expected from online Zooniverse participants and to identify the differences that may be expected for an in-gallery experience. This first test helped to shape the project’s future steps. The initial reactions on the floor were reassuring. Our guests found the activity to be self-explanatory, easy to follow, and intuitive. Many expressed a profound interest in being able to interact with museum objects in the digital space and at a more personal level than they could within the display cases on the museum floor. However, the original beta test had some limitations: the prototype only tested the Yes/No swipe workflow, meaning the responses were limited to opinions on the simplest task, not more challenging ones like drawing boxes or identifying constellations. We held the testing at a Members’ Night, meaning that the responses came from our most excited and loyal audience. The next step for the team was to create a more in-depth prototype that could be used not only at future Adler events, but also in tests on the museum floor with everyday guests. The goal of these prototypes and beta tests were to work out what, if any, difference there might be between the reactions of our in-gallery guests to crowdsourcing projects and the online

members of the Zooniverse community. We also want to determine simple but still unknown answers to questions such as:

1) How long will a guest participate in this interactive before moving on? 2) Is there one task (segmentation, identification, etc.) that is more popular among guests

than others? 3) Are each of these individual workflows presented in a way to allow for an intuitive

experience for our guests? Testing on the museum floor over a three-month period provided our team with solid next steps. We quickly realized that our simplest Yes/No swipe workflow was the most well-received (“Do you see more than one constellation? Select Yes or No”). However, it did not help to give us useful data, as the Adler catalogue data already listed which images featured single vs. multiple constellations. Instead, we chose to rework this easy and popular question style to help us identify multi-constellation images that needed to be cropped. As we tested the segmentation workflow (“Draw a box around each individual constellation in the image”) we noticed that there was a certain point at which guests would become overwhelmed by the task: guests were willing to draw boxes around 4 constellations before giving up and abandoning the task. Based on this feedback, we updated the Yes/No swipe workflow to help us identify which images contained more than four constellations. Any images containing more than 4 constellations would need to be cropped down by the Collections staff before being sent to the segmentation workflow. Similarly, as we tested on the floor we realized that many of our guests were too intimidated to try either workflow if staff members were standing with the iPads observing guests. There was a perception that the members of staff facilitating these tests were ‘experts,’ which made many of our guests feel wary of participating. We often heard them vocally expressing concerns of being ‘wrong’ or questioning their knowledge of constellations. Many were convinced they did not have the skill set or knowledge required to complete these tasks. Notably, children under the age of 10 did not seem to have this bias and would immediately jump into the tasks without hesitation. It was older teenagers and adults who required reassurance, and even then, would often leave the testing without completing a classification due to fear of being wrong. Outcomes and Continued Development Running these tests on the museum floor helped us re-evaluate our question and task design. Originally, we planned for the identification workflow (that would provide guests with a series of questions until they worked out which constellation they were viewing) to be included in the Chicago’s Night Sky exhibit. However, as noted above in the discussion of the workflow design process, the Zooniverse mobile app does not yet have the functionality required to present a series of yes/no swipe options based on a single image. The desktop version of the Project Builder does, however, include this capability, which fueled the decision to offer that workflow to online volunteers on the Zooniverse website. Furthermore, our team reasoned that an online component will be a good follow-up task for guests who enjoyed the experience on the museum floor and want to contribute further.

Similarly, the testing process taught us that it is crucial to provide contextual information about why this project is taking place. All online Zooniverse projects feature an ‘About’ page, which allows teams to explain the research and motivations behind their project, and an in-museum crowdsourcing experience similarly merits research justification and contextualization. During testing, we found that guests were more engaged and willing to participate when they were told 1) why we were asking them to take part; and 2) that they were helping with real, ongoing research. Since the in-exhibit interactive will not be supervised by staff, we decided that having an introductory video with this information would help attract our guests and allow us to give them the background information on constellations that they requested during testing without the time constraints or potential barriers to participation that may result from including large amounts of instructional text in a gallery space. This video will provide the research background for the project, while an in-app tutorial will help guests understand how to use the application. Many guests who participated in testing also expressed an increased interest in the Adler collections. After engaging with the digital images on the interactive, many then wished to view the objects on the museum floor. This feedback helped with the Chicago’s Night Sky exhibition planning, putting an emphasis on including some of these celestial cartography pieces within the exhibition and pointing to them from the interactive using messaging like “Find me in the museum!” Signage around the objects on the floor will also direct visitors back to the interactive. Testing for Mapping Historic Skies is ongoing and will continue through the spring of 2019, with an anticipated final launch to coincide with the opening of the Chicago’s Night Sky exhibit in November 2019. The core idea behind this exhibit is that the sky connects us all as humans and in our communities, and we believe Mapping Historic Skies will provide our guests with a unique opportunity to participate in the creation of a curatorial research tool, while engaging with cultural artifacts that express an enduring connection to the sky that is both historical and global. As the Adler Planetarium founder Max Adler said at the institution’s opening in 1930: “We dwell too little upon the truth that under the heavens everything is interrelated, even as each of us to the other.” (Adler 1930). This is a message we are proud to continue to share. Conclusion The purpose of this article is to give insight into the process of designing and prototyping a crowdsourcing project for use within museum exhibits as well as on an online platform. We highlight the tools available through the Zooniverse Project Builder, but also focus on the importance of in-person testing and using guest feedback to refine workflow design. We are still in the process of testing Mapping Historic Skies, and will continue to do so in advance of our planned November 2019 launch. We hope that the discussion of the development and prototyping process will shed light on the importance of iterative design and incorporating guest feedback, and help other teams who may wish to include a crowdsourcing project in their museum space.

Acknowledgements This article uses data generated by the Zooniverse platform, development of which is funded by generous support, including a Global Impact Award from Google and by a grant from the Alfred P. Sloan Foundation. We gratefully acknowledge the Zooniverse web development team based at the Adler Planetarium in Chicago and the University of Oxford, the international research teams who lead Zooniverse projects, and the worldwide community of Zooniverse volunteers, without whom none of this would be possible. References Adler, M. (1930). “Speech at Adler Planetarium and Astronomical Museum Dedication.” Speech. The Adler Planetarium, Chicago, Illinois. May 30, 1930. Cavalier, D. (2012). “Announcing the SciStarter kiosk interface for schools, museums and public areas.” SciStarter blog: Citizen Science Projects, People, and Perspectives . Published December 13, 2012. Consulted January 8, 2019. https://blog.scistarter.com/2012/12/announcing-the-scistarter-kiosk-interface-for-schools-museums-and-public-areas/ von Konrat, M., T. Campbell, B. Carter, M. Grief, M. Bryson, J. Larrain, L. Trouille, et al. (2018). “Using citizen science to bridge taxonomic discovery with education and outreach.” Applications in Plant Sciences 6(2): e1023. Consulted January 8, 2019. doi:10.1002/aps3.1023 . Lintott, C., K. Schawinski, A. Slosar, K. Land, S. Bamford, D. Thomas, J. Raddick, R.C. Nichol, A. Szalay, D. Andreescu, P. Murray, L. van den Berg. (2008). “Galaxy Zoo: morphologies derived from visual inspection of galaxies from the Sloan Digital Sky Survey.” Monthly Notices of the Royal Astronomical Society 389 (3), 1179-89. Trouille, L., C. Lintott, G. Miller, H. Spiers. (2017). “DIY Zooniverse citizen science project: engaging the public with your museum’s collections and data.” MW17: MW 2017 . Published January 30, 2017. Consulted January 8, 2019. https://mw17.mwconf.org/paper/diy-your-own-zooniverse-citizen-science-project-engaging-the-public-with-your-museums-collections-and-data/