CASE CARDS

AT WHICH STAGE OF THE VISITOR JOURNEY CAN VISITOR DATA BE COLLECTED AND USED IN A MEANINGFUL WAY?

The Australian Centre for the Moving

Image

Who ACMI (AU) What ACMI Soft Power Map, visitor journey map

Where Melbourne, AU When 2018

Description ACMI demonstrates an all-embracing approach to visitor research and endeavours targeted at outreach expansion. In terms of global outreach and enhancing the potential of travelling formats and collection’s mobility ACMI’s Soft Power Evaluation Tool (based on Geo-Spatial Museum Data) is of particular interest. In 2018, in collaboration with University of Melbourne, ACMI introduced a pilot web application, ACMI Soft Power Map, a focused, single-museum online tool to geo-visualise and assess the museum’s “attraction power” in Melbourne and abroad. Context The Australian Government stressed in the Asian Century White Paper (2012) the growing economic power of Asia and urged the country to rethink the national strategic objectives for the next decades. Particularly striving to create deeper connections with Asia, to broaden the flow of ideas and exchange of knowledge and capabilities. The Deep Mapping research project was initiated to comprehend, evaluate and leverage the Australian soft power generated and facilitated by Australian cultural institutions. The project established a collaboration with the Australian Centre of the Moving Image (ACMI) and designed a pilot ACMI Soft Power Map - a focused, single-museum online tool to geo-visualize and assess the museum’s “attraction power” in Melbourne and abroad. The platform instrumentalises geo-visualisation, data mining, digital storytelling and data-curation techniques to explore correlations in geographical layers of museum data, including between collections, online and onsite audiences, international activities and constituencies. Tool ACMI Soft Power Map is a digital platform which “exposes and explores correlations in geographical layers of museum data on several levels, including collections, online and onsite audiences, international activities and constituencies”. The mapping system consists of five key layers “that allow users to evaluate ACMI power on the soft power conversion scale from mere resources, to strategic outputs and, finally, to a target response”. Specific layers and indicators the Soft Power Map are:

− Collection Appeal Power (presenting potential appeal of the collection to people living in the certain country and comprising indicators on digital mobility, physical mobility, collection diversity, digital diversity);

− Online Engagement Power (representing digital visitors and followers from a certain country and comprising data on online visitation, social media engagement, experience reflection);

− Global Connectivity Power (diversity, geographic spread, durability and strength of institutional connections established in a certain country);

− Local Engagement power (attraction power of a travelling exhibition in a certain city). * In the last decade, ACMI’s international shows travelled to 16 cities across the Asia-Pacific region, Europe, and the Americas. The tool additionally provides Melbourne engagement power indicators (citizenship, English speakers, indigenous population, income, etc.).

In focus is ACMI’s collection. The tool presents a short overview of the collection’s connection to a certain country and the metrics based on engagement level (potential) in the respective country. Collection Appeal Power Layer maps the cultural and linguistic diversity of ACMI’s collection, which includes around 200 thousand films, 70% of which were produced outside Australia. Movies produced in 49 different languages make this content accessible to people living in over two hundred countries around the world. Further collection characteristics are correlated with multiple variables pointing to the social demographic profile, physical or digital mobility of population of specific countries. This allows to evaluate collection’s potential appeal on the global map to get a better view of what strategies should be used to engage international visitors. The layers allow to access an immense number of visitor data on both global and local level. The digital mapping system exposes the geographic origin of ACMI audiences on each of the above layers. In local terms, it provides a detailed view on how ACMI visitors are distributed across over two hundred post code areas in the city, shows key data on each neighbourhood in terms of social demographic profile, cultural, religious and linguistic diversity, family, household, professional affiliation, and income information. Another layer, Global Connectivity Power Layer, measures diversity, geographic spread, durability, and strength of institutional connections established between ACMI and around 180 organisations in 80 cities across the globe. To suggest an example, ACMI developed a series of blockbuster exhibitions in collaboration with one of the largest Hollywood producers, DreamWorks Animation. These travelled across Asia-Pacific in 2015-2017 and were hosted by a number of museums, such as the ArtScience Museum in Singapore, the Te Papa Museum in New Zealand, the Seoul Museum of Art in South Korea and the National Taiwan Science and Education Centre. The recently completed focused research investigated and explained local and global mechanisms of “attraction” power generated by ACMI blockbuster in different Asian countries. When compared across three cities — Singapore, Seoul and Taipei — it was revealed that Singapore enjoys stronger global connectivity and exposure and has larger and more diverse onsite and online audiences in the ArtScience Museum. Local Engagement Power Index, when correlated to data across four layers of ACMI Soft Power Map and in comparison to South Korea and Taiwan, showed that Singapore has much higher indicators for ACMI Collection Appeal, Online Engagement and Global Connectivity. The high Online Engagement Index proves that multilingual and multicultural Singapore, with its high internet penetration rate, is well placed to establish multidirectional digital links with ACMI through numerous online communication activities.

Moreover, ACMI has already established strong relationships with key local cultural players, such as museums and festivals, and achieved strong brand recognition. This in return provided a background for the blockbuster to engage and attract local audiences. With all these factors and research findings, it goes without surprise that the ACMI blockbuster exhibition “Wonderland” was to start its global tour in April 2019 precisely from Singapore. Also in May 2019 a new layer was developed, the Local Engagement Power Forecast, which forecasts the soft power and attendance numbers of the traveling “Wonderland” in potential hosting cities across continents. Among 17 potential cities where “Wonderland” might travel in the next decade is a span from Los Angeles to Taipei. Just to continue the line, Singapore is again a leading city in the Asia Pacific with the highest Local Engagement Power Forecast. But not only in Asa Pacific, it has actually outperformed Berlin, Paris and Tokyo among other cities. Visitor data and Wonderland ACMI’s “Wonderland” exhibition can be seen as exemplary in regard to the attention paid to the visitor journey optimisation. The exhibition was designed to reference Carroll’s book and according to Lucie Paterson, “to be a little like maze, full of surprises”. Thus, ACMI’s team introduced a special element to the exhibition - a map, meant to accompany visitors on their journey and guide them through the exhibition. As mentioned in Lucie Paterson’s article, “the map itself had to achieve a lot for the exhibition including additional information for deep divers, basic wayfinding, exhibition branding, a post-experience call to action, character representation, personalised projection triggering, visitor tracking and even a hidden riddle for the most curious to solve.” The map served also as a link to the post-visit experience giving access to visitors creations produced in the activity space of the exhibition. And overall, “the map became the tool to learn more about visitor experience, especially in view of [ACMI’s] new space opening”. With a number of smart solutions applied throughout the visitor journey, the exhibition demonstrated how the data can be collected at each stage of a visit and effectively improve visitors’ experience.

As seen in in the table above, pre-visit data was collected with the help of Google Analytics and meant to answer a number of questions concerning booking (what device was used for booking, was this the first booking or not, was the ticket booked for a specific time or event, etc). The onsite data was collected with Map OS and was aimed to identify the duration of the stay, time and place of checking in and more. Finally, the collection of post-visit data was also assisted by Google Analytics and was meant to monitor the reasons behind visitors’ decision to login, the preferable content and willingness to sign up for the informational letter.

Post-visit visitor research As revealed in Shelley Matulick, User Experience Researcher and Content Producer at ACMI, explains, the research was aimed to understand how visitors used the Lost Map of Wonderland in the exhibition, their readiness for a post-visit experience and how the institution could motivate them to extend their experience of the online products. Through NFC analytics, Google Analytics and observational research of the 94% of visitors (who were given a map) it was found that:

− 88% tapped on more than one scanning station (six of which available in the space); − the average visit duration to the exhibition was 62 minutes; − 10% visited the post-visit website; − 50% of those got to the very end of the website; − the most popular content was the Making Wonderland video at the end of the website with 68% watching all 8 minutes; all

embedded videos were more popular than video links and articles; − interestingly enough, post-visit website visitation followed the same pattern as exhibition visitation — higher at weekends, public

holidays and late night events.

The above findings were enriched by a series of one-on-one interviews with a variety of visitors. More than100 visitors were interviewed. As Shelley Matulick mentions, “the interviews were accompanied by the “lo-fi, fast-turn-around videos that could be shared with our US-based exhibition design firm, Second Story, and internally at ACMI in the same day”. Interviews ranged from 3 to 30 minutes. The next stage of the research was creating visitors’ personas to “illustrate the revelations” and instrumentalise the learning for future ACMI’s products. Some of the Personas:

− cultural segment persona: Affirmation, needs: new information in a fresh easy to digest format; − culture segment persona: Entertainment, needs: strong visual storytelling; − cultural segment persona: Entertainment, needs: interactive experiences, gameplay.

Visitor interviews were not the only way ACMI gathered qualitative data from visitors. The institution also utilised:

− Slack “visitor feedback” channel where institution’s visitor services officers give day to day feedback from the floor; − Trello board that is open to all the departments to contribute to.

Organisation-wide open access to research, both process and findings, has changed the attitude and outcomes of its undergoing. All team members could contribute and a constantly iterating research plan was ensured. It also prioritised research within teams’ agendas, focused the endeavour (beyond the plethora of possible scenarios and solutions demanding research-based approach) and informed management how and why decisions were made to invest time and effort.

Benefits − an innovative tool generating an extensive amount of data and not restricted by geography; − a tool fully integrated into the exhibition allowing to collect visitor data on-site and at post-visit stage. With the Australian Government seeking innovative and reliable approaches and tools to measure its soft power, the ACMI Soft Power Map offers a robust platform to address these tasks. By mapping soft power, we gain new insights into how the global museum sector works. For example, once the global connections between institutions are mapped it becomes clear that museums amplify their soft power when they work together and establish meaningful connections, rather when acting as rivals. The next stage of the project is to advance the mapping system to measure soft power of museum clusters and cultural infrastructure within cities.

Additional info & comments

Wonderland exhibition awards 2019 GLAMi’s Best in Show award 2019 GLAMi Exhibition Media or Experience: Immersive Category award

Involved Parties University of Melbourne (ACMI Soft Power Map)

Contacts ACMI: Lucie Paterson Head of Experience, Product & Digital lucie.paterson@acmi.net.au Seb Chan Chief Experience Officer seb.chan@acmi.net.au

One Further

Who One Further (UK) What Website analytics

Where London, UK When since 2014

Description One Further is a digital analytics and user research consultancy working in the areas of ecommerce, culture and higher education. One Further worked with major cultural institutions helping to provide insights across collections, visitor experience, marketing, membership, ticketing, ecommerce, image licensing, apps, interactives and publishing. One Further offers the following services: Analytics planning and implementation

− carrying out analytics audits and fixing serious data quality issues; − upgrading legacy website analytics, moving tags into Google Tag Manager to take advantage of additional functionality and

flexibility; − working with developers to make websites more easily trackable; − developing KPIs and metrics to align with organisational and departmental objectives; − providing marketing and ecommerce teams with clearer information about the impact of their campaigns; − helping teams to make informed decisions around content publishing, including blogs and social media.

Dashboards and reports

− producing a variety of reports, from simple dashboards giving departments information about website traffic to their pages to more advanced reports that collate stats from multiple sources.

Training One Further provides tailored training to those museum teams which aim at having a better understanding of digital analytics tools such as Google Analytics and Google Tag Manager. For some of museum clients One Further provides a flexible, on-demand analytics service. 90% of One Further’s clients are cultural institutions, and many of the company’s other clients are nonprofits and membership organisations. Unveiling the specifics of cooperating with cultural organisations, Chris Unitt, One Further’s founder, highlights that there are two main ways they work with cultural institutions:

1. Giving institutions the tools to understand their audiences themselves. This may involve setting up Google Analytics for them properly and advising on other techniques they can use and creating custom reports and dashboards to make their data easier to access.

2. Working with them to answer specific questions.

There's a great variety of questions One Further might be asked to help answer. For instance: − who is accessing our content and how can we make sure we create content that will resonate with them? − how can we make the process of browsing our site easier? − we're going to launch some new functionality, how can we see if there are any problems with it before it launches? − how can we make our checkout process smoother and increase conversions?

To answer these questions, the company uses a variety of tools, depending on what's most appropriate.

Types of online visitor data and technological tools One Further works with:

− marketing: which channels people use to arrive on the website; − website functionality: what features do people make use of, do they encounter any errors while browsing the site, can people

complete tasks on the website quickly and easily; − content performance: which content is popular, are people encouraged to take action after reading/viewing something.

Most frequently used tools include Google Analytics, Hotjar, Treejack, website surveys, and in-person/remote user testing. Organisations should have a foundation of a quantitative analytics tool (like Google Analytics) and this should be implemented properly - collecting pageviews, other interactions and transactions. This should then be augmented with periodic qualitative feedback aimed at understanding:

− the general health/perception of the website and who's visiting it; − attitudes towards specific types of functionality of user journeys so that they can be improved.

Examples of how the insights of the research have (or could have) informed/shifted KPIs of cultural institutions Chris Unitt: “Quite often we find that organisations measure things that don't really matter. Google Analytics shows metrics like pageviews, bounce rate, pages per session, and average session duration. We find that these (especially presented for the website as a whole) are near useless, and sometimes misleading. We get the organisations we work with to focus on what their specific objectives are and to measure how the website is helping people to achieve those objectives.” Challenges Challenges tend to come when:

− there's an issue with the technology – especially 3rd party systems (for ticketing/donations/memberships) where One Further can't add tracking codes and so only have a partial view of audience behaviour;

− institutions want answers that the data doesn't lend itself to answering. Two classic examples: a) how does our website impact on visits to our institution where entry is free and not ticketed and b) what is the “impact” of our website content, when that impact is very subjective and doesn't manifest itself through clicks on the website (e.g. someone is inspired to get off their computer and pick up a paintbrush).

Audiences comprehension: − using online collection on museum’s website for very different purposes to what was expected (e.g. researching family history, or

getting inspiration for costume design); this has influenced redesign projects to make those sites less academic in nature and cater for those other use cases (3D images of artworks in the online collection, etc);

− large numbers of people encountering errors on a website which previously the institutions weren't aware of.

One Further’s grow-into areas regarding visitor data aggregation and analytics: − institutions that only have a partial view of how people are interacting with them online; given the increasing importance of digital

channels for attracting audiences and delivering an institution's activity, the company expects more people to take this aspect of their audiences more seriously;

− increasingly working internationally (particularly in the US).

Future-proof technological developments: − a new version of Google Analytics that offers new possibilities for understanding audience behaviour; − how changes in privacy-focused legislation and browser technology will impact on the ability of institutions to make sense of how

their audiences interact with them.

Services provided to specific museums British Museum – digital analytics, search engine optimization, training. Victoria and Albert Museum – digital analytics support, training. Royal Museums Greenwich – digital analytics support, strategic consultancy. Royal Academy of Arts’ case study One Further worked with the Royal Academy of Arts through their development of a new digital content strategy assisting in turning their objectives into KPIs and creating a dashboard to track performance. In 2018, the Royal Academy of Arts took on a project to modernise the website and social media content with the aim to target new audiences. According to the overview of the case, published in 2017, in practice, the process meant stopping everything the Royal Academy publishes online and starting again – “this time with [their] values, objectives and audiences at the core of everything [they] do”. The challenge behind the project originated from the fact that out of 365 pieces of editorial content produced by the RA, only 65% of it had been viewed less than 500 times (to compare, institution’s most viewed article has been read over 70,000 times). Additionally, the researched showed that the Royal Academy’s social media weren’t performing well despite high resources put into them. As part of transformation, the Royal Academy’s overall digital strategy focused on four objectives: to grow the audience, to deepen the relationship with them, to convert them into customers, to retain those relationships and customers and to make sure that every piece of content was meant to achieve at least one of those four things. As for the targeted audiences, previously, the RA had aimed at its three key target audiences: existing core audience and two more slightly younger.

With these new types of potential visitors in mind, the RA prepared a series of charts depicting each audience-group’s interests and the RA’s values as an organisation. Each piece of content had to be oriented at one of these three audiences. As a further step, the RA’s previous content was checked against these charts. The comparison showed that “much of the previous content catered for the institution’s core audience, some of it catered for no-one, and none of it catered for those two, younger prospective audiences of the RA”. Among the solutions there were:

− a strong set of new article and video formats, each targeted at one of specific audiences and each with a plan of how to help that content find that audience;

− some series developed with departments who would take ownership of them.

From a technology perspective, the institution underlines that it was important that digital skills were spread throughout the organisation. Thus the RA signed up to a new social media platform, Percolate, that gave the institution “unlimited logons and had approval mechanisms”. Additionally, the RA “rolled out training to their CMS so that departments could build their own content, and … arranged analytics training to encourage interest in the outcome of every piece of content”. One Further joined the project at the further stage of creating a new dashboard which helped to identify KPIs for each objective the RA set and to track them. According to the RA, a summary page gave them “top level stats against a target for each month, and a page for each objective tell[ing] which pieces of content are contributing to each”. The first dashboard was focused on tracking the editorial content and later followed by a website-wide dashboard for the same four objectives. According to Louise Cohen and Amy Macpherson, the above changes have made a huge difference to both, the institution’s output and culture. The AR got reminded “to be more focused with where [they] put resources, and to push for continual improvement – and importantly, to keep sharing those learnings across the organisation”.

Benefits − digital analytics strongly leaning on audience’s culture segments and directed at identifying meaningful KPIs

Additional info & comments

Projects One Further currently collaborates with: Culture Restart Toolkit, a survey across multiple cultural institutions to understand their audiences' interest in attending their venues and their attitudes towards accessing digital content; Digital Heritage Lab, a free programme for small and medium heritage organisations seeking to develop their digital capabilities and capacity, funded through the support of The National Lottery Heritage Fund as part of the Digital Skills for Heritage initiative;

Culture 24 and their Let’s Get Real 10 Years On: How to evaluate online success.

Contacts One Further: Chris Unitt Founder chris@onefurther.com

Natural History Museum

Who Natural History Museum (UK) What Scalable Power BI and custom visualisation solution

Where London, UK When 2019

Description The Natural History Museum is the most visited natural history museum in Europe and the top science attraction in the UK. The museum welcomes around five million visitors each year while its website receives over 850,000 visitors a month. In 2019, the NHM partnered with Microsoft and, consequently, data analytics company Simpson Associates to create the museum’s Digital Twin Technology Vision enabled by the Power BI. The primary goal is to establish a central data hub that brings together all of data sets and systems in a coherent way. The cooperation is aimed at ensuring the NHM’s evidence-led and data-driven orientation through unlocking and organising data in order to help employees understand and better devise plan scenarios. The data insights were seen as an instrument to understand how people use spaces, the condition of museum’s unique collection of 80m specimens and how to run and maintain its estate sustainably. However, to make these insights actionable, the NHM needed a solution to provide all employees with secure data access. As the NHM’s team underlines, the solution was meant to save time from making reports to actually act on them. The transformation started from the Retail department which generates around £2M a year to support the museum’s collections, buildings and research. The main challenge which the department aimed to overcome was manual consolidation of data using complex spreadsheets from different sources and constant debate on the validity of numbers. This approach was immensely restricting in terms of time and ability to drive actionable insights. The need for a solution was crucial not only to specific departments but for the museum at large. NHM is a complex organisation with a lot of data sources (incl. 15,000 sensors used to manage the vast 100,000m2 estate). Such an immense amount of data has to be turned into cohesive visual reports and dashboards to help the museum optimise manual processes, improving productivity and gaining deeper analysis methods, which can eventually be shared throughout the organisation. According to the museum team, the transformation is accompanied with the certain inter-organisational challenges:

− making all the departments start using the new model; − unifying the approach to data between all the departments; − overcoming the affinities to data.

Microsoft Power BI visualisation The second area of focus for NHM’s transformation is Estates, who found its building survey data difficult to access and evaluate since “thousands of lines of conditions data and estimated repair value for each space within the museum’s 100,000 square meters were hidden in a siloed system”. The solution would be in finding a way to present data in the context of a site map to assess repair value and help decide how to invest limited resources into improvements. By demonstrating that CAD vector site drawings can provide a dynamic 2D representation of the museum in Power BI, Simpson Associates showed the museum departments how to see where and what type of repairs are needed and make the repair value available per building, floor and room. This new streamlined process works regardless of whether staff are maintaining one area or combining repairs across spaces. Such an approach is saving time, increases efficiency and can inform management on how to reduce energy use and meet the museum’s sustainability goals using NHM space data presented in an impactful way.

Examples of dashboards

Plans The NHM plans to expand visualisations across the museum and look for wider datasets to incorporate into the model; such as pest management, occupancy, collections locations and environmental monitoring data. Among other grow-into areas is ticketing. Before the Covid -19 prevention measures imposition, the NHM had free access to the museum. However, the introduction of timed tickets has impacted the museum’s data collection processes. In fact it proved to be beneficial for the revenue streams optimisation. The museum introduced new donation options to be displayed when booking an entry slot. The donation points were introduced and were highly productive in encouraging visitors to actually support the institution. Alongside time slot, new add-ons were introduced such as, for example, reservation of retail items. Results the solution brought:

− deployment of an automated reporting and analytics platform; − saving Retail 10 hours of reporting time per week; − removing all risk associated with manual reporting; − providing confidence in consistency and accuracy of data.

Adoptability of the system Simpson Associates: “The custom visualisation of floor plans we developed for the NHM is a repeatable solution that can be adopted by industries such as retail and manufacturing where optimisation and floorspace management is vital for success. By visualising data related to space performance, retailers can transform the customer experience and optimise return on space. While manufacturers can see how production might work in any situation, how certain setups might impact operations and compare alternatives. Floor plans combined with data enriched with spatial context can play a vital part of the journey towards complete automation, because of the way it provides insights which are impossible through simple bar charts and tables. The ability to present data in an easy to use format for all employees increases engagement and empowers fast decision making, critical for industries where time means money.”

Benefits − a solution allowing the NHM to unify data in a cohesive visual way and as a result improving predictability and decision-making processes;

− the subsequent digital transformation: front-end user journey improvement, implementation of new revenue channels, etc.

Additional info & comments

On-site visitor research The NHM’s Audience Research team works to continually improve the museum’s visitor experience and ensure that the museum's activities benefit a wide range of audiences. The museum carries out evaluation for most of its exhibitions, activities and programmes and works collaboratively with partners to evaluate the findings. According to the NHM, consulting directly with the visitors “helps inform the institution's strategic planning using an evidence-based approach”. The evaluation studies take place at different stages of exhibitions, programmes and digital activities, and encompass three main types of studies:

− front-end evaluation (occurs during the development phase of an exhibition, event or other activity and helps “to gauge audience interest levels and prior knowledge about a subject”; this type of evaluation contributes to “developing stories, goals, communication messages, learning outcomes and interpretative strategies”);

− formative evaluation (is held during development and production phases and helps to test exhibition components (using prototypes) such as text, instructions, graphics and usability, as well as the specific communication messages and learning outcomes; this type’s findings are incorporated into the project);

− summative evaluation (is held during the exhibition and is aimed at understanding “the learning that has taken place and the impact of specific design features”).

Involved Parties Microsoft Simpson Associates

Contacts Natural History Museum: Richard Hinton Head of TS and Enterprise Architecture Planning/Technology Solutions r.hinton@nhm.ac.uk

Nubart

Who Nubart (ES) What Audio guides

Where Barcelona, ES When since 2016

Description Nubart is a browser-based progressive web-app (PWA) which is accessed through the specially-designed cards that museums visitors can receive upon their arrival to the venue. Thus Nubart’s audio guide works as a non-transferable key that allows easy and quick access to multilingual content directly from the visitor’s smartphone when the code from the card gets scanned. Among Nubart’s products which can be combined within the card, there are audio guides, group guiding system, audio-video synchronization and more. Generally, Nubart offers a modular system within which all products can either be offered separately or integrated in one single Nubart-card. Nubart’s guides can be used for data collection purposes and retrieve the following types of data:

− home country and the native language of the visitor; − visitors per day and by time-of-day; − duration of the visit and minutes played; − ranking of most-listened-to tracks; − number of files accessed by the visitors; − the type of devices used (Android, iOS); − first-time / returning visitors.

Nubart’s team underlines, that the amount and quality of gathered data can effectively serve predictive purposes. The longer an institution uses cards, and the more cards a venue distributes, the more precise the provided analysis can be. However, even small samples allow the company to detect trends regarding contents and visitor behaviour. Furthermore, since the beginning of operation, Nubart’s reports allow for some degree of predictability. For example, nationalities by the time of the year: if the reports tell the venue that in March 2018 it had X% more Japanese visitors than in January, the institution can predict similar affluence of Japanese visitors for spring 2019. Additionally, working with multiple organisations and aggregating the different sets of data enables Nubart to make predictive analyses of the industry in general. Visitor journey Nubart aims at collecting data at each stage of the visitor journey. The post-visit data is collected in the format of the integrated feedback form. Besides, at the post-visit stage, Nubart collects the aforementioned types of data: downloaded files, browser language and country, and so on. These particular types allow to measure the reliability of data gathered from visitors’ feedback forms. A case that can illustrate the above-mentioned post-visit engagement is the exhibition “Stanley Kubrick” at CCCB (2018 –2019). The graph below shows the number of files accessed while the show was on display. From the diagram, it can be inferred that visitors were highly engaged at the post-visit stage since an extensive number of files was accessed when the venue was closed (before 10 AM and after 8 PM).

Another recent example is the “Albrecht Dürer” exhibition in Albertina, Vienna. The following graph represents the number of users that accessed the audio guide once the exhibition had already finished, from 07.01.2020 onwards, and confirms that Nubart’s cards extend the lifespan of the audio guide’s contents.

In regard, to post-visit services, the access to contents from Nubart’s guide does not expire after the visit is ended. All the contents of the guide can still be checked again later, even years after the visit, depending on the customers’ setup. Data’s availability to institutions Currently, Nubart provides a CSV file with the usage of cards per month and more complex reports as a PDF. In a very near future, Nubart is to set up Mongo-Charts that can be made available to third parties. Usage of data Nubart has identified a wide range of ways in which their customers use the data provided:

− to explore the viability of outdoor routes according to visitors’ interests; − to confirm whether visitors follow, or not, the recommendations of institution’s on-site staff regarding POIs and highlights; − to identify which subjects visitors are most interested in (based on the feedback and the most played tracks); − to fix the mistakes and to improve communication with the visitors (occasionally some visitor notice a mistake in the information

provided and bring it into attention in the feedback form); − to identify which stops work better or worse and to adjust the contents accordingly; − and generally, to know visitors better (their profiles and interests, and to segment them accordingly) to make a better-informed

strategic decisions.

Sector-specific insights Nubart provides service not only to museums or exhibition centres, but also churches, parks, open-air urban and natural itineraries, as well as trade fairs and even cemeteries. This allows Nubart compare data from different sectors and outline sector-specific nuances. For example, Nubart highlights a very common issue for most cultural venues, so-called museum fatigue. This phenomenon is connected to the visitors’ itinerary, going through which they inevitably lose their interest in the museum content. Even though in some cases the drop is less evident than in others, it always happens. The following graph provided by Nubart shows how aggregated data from 3 different exhibitions held at the same venue (one dedicated to quantum physics, an art exhibition and an exhibition dealing with the theme of cinematography) proves that even with variations, all 3 exhibitions evidence the visitors’ museum fatigue. The only thing that changes is how sharp this fatigue gets (quantum physics exhibition shows a more intensive drop that the one on cinematography). The graph also helps to understand at which point of the experience this

happens (the drop is more sudden during the first half and tends to even up towards the end).

Nubart’s grow-into areas in terms of visitor data collection and analysis

− introducing a dashboard powered by Mongo-Charts which will enable the institutions to check their metrics and learn about their audio guide’s performance and their visitors;

− enhancing and expanding statistics and graphics; − making the most of the feedback forms and adapting them to provide a better audience segmentation; − creating an admin-dashboard with “open data” available for research centres; − preparing reports about trends in museums and other attractions.

COVID-19 and data collection Some of Nubart’s features, which now can be considered in the context of COVID-19 prevention measures, were already in place long before the pandemic. Among them there is a hygienic advantage of Nubart’s card as opposed to portable on-site devices. Also, some other features that before COVID-19 were considered as aged, like QR codes, are now proven to be extremely relevant (e.g. menus in restaurants). Regarding new developments, as a result of the pandemic, Nubart developed a group guidance system that allows having large guided groups with a safe distance between all participants ensured. The voice of the guide is transmitted to the group via the Internet (Wi-Fi or mobile data). Thus the code on the card allows to turn the guide’s smartphone into transmitter and the participants’ smartphones into receivers.

Nubart’s observation is that COVID-19 increased the usage of BYOD technology. This positively impacts the demand in Nubart’s products and allows the company to collect large amounts of data and make better conclusions about visitor behaviour.

Benefits − Nubart’s guides do not require costs of traditional devices (storage, sanitisation, charge batteries, repairs, picking up at the exit); − they do not require the submission of emails and personal data; − ensure higher take-up rate than apps; − ensure access to real data during the visitor experience in-situ, plus any subsequent post-visit card usage; − provide wide variety of supported languages (including RTL languages); − modular system, each with specific features (including expiry dates for some modules), allows to organise the content and to

provide very different services in one single card; − provide an option to be combined with ticketing, audio-video synchronisation, group guiding system and more; − offer wide variety of multimedia elements that can be integrated (live streaming, content tags, donation buttons, transcriptions for

the deaf, subscription forms, and more.

Limitations Nubart’s audio guides are not available on-site for people without a smartphone.

Costs & Timeframe With the content already provided, the timeframe to implement the audio guides within the institution is around 4 weeks. If the content is to be produce, the timeframe will vary according to a number of factors. The cost of Nubart’s basic offering (consisting of 3,000 cards, without production) is 4,000€ (that is 1€ per card plus a setup fee of 1000€). In the case of larger organisations, it depends on the number of cards, since Nubart offers discounts for large quantities. Other issues to consider are whether the cards are given out for free to every visitor or sold apart. If Nubart is to produce the content as well, then the quote will depend on a variety of issues that need to be discussed specifically in each case. However, the company offers revenue-share agreements to organisations with vast volumes of visitors.

Additional info & comments

Facts & figures : − Nubart has been producing audio guides cards since June 2016;

− Nubart currently works with +50 customers; − over 125,000 people have accessed Nubart’s audio guides so far; − Nubart’s cards have been used by visitors coming from a wide variety of countries.

Involved Parties Among Nubart’s clients: Albertina (Vienna), CCCB (Barcelona), Museo de la Nación (Lima), Castle Bourscheid (Luxembourg), Museum Angewandte Kunst (Frankfurt am Main), Museu Faller de València (Valencia) and many more.

Contacts Nubart: Rosa Sala Founder rosa.sala@nubart.eu Marcela Rosemberg Chief of Customer Happiness marcela.rosemberg@nubart.eu

Cleveland Art Museum

Who Cleveland Museum of Art (US) What Art Museum Digital Impact Evaluation

Where Cleveland, US When 2016-2018

Description In 2016, the Cleveland Museum of Art’s Research and Evaluation team in cooperation with research, evaluation and consulting company Rockman et al launched a two-year project to examine new ways of measuring the impact of the digital interactives in art museums. Project funding was provided by the National Endowment for the Arts. The Cleveland Museum of Art underlines its background which served as a solid and favourable basis for this research: “With its award-winning ARTLENS Gallery and comprehensive digital strategy paving the way for digital innovation in art museums, coupled with an established, robust visitor research program, the institution was well positioned to take on this work.” ARTLENS Gallery is an experiential space that uses innovative technology to put the visitor into conversation with works of art, encouraging engagement on a personal, emotional level. It comprises interactive spaces such as ArtLens Studio, ArtLens Exhibition, ArtLens Wall and more, all of which help to explore the museum’s collection in unique ways, “whether it be through learning art elements to observe in the galleries, discovering new artworks via the games, looking closer at artworks, or curating an experience by building a tour or a list of favorite works”. ARTLENS Gallery opened in June 2017 (following its predecessor, Gallery One) and became a test site for developing new metrics for exploring visitor engagement. Cleveland Museum’s “Art Museums and Technology Developing New Metrics to Measure Visitor Engagement” paper extensively describes the research undertaken by the museum to measure the impact of digital interactive experiences on visitor engagement. After a comprehensive literature review and series of stakeholder interviews, the CMA and Rockman team developed a semi-structured, cognitive interview protocol to test out concepts and language surrounding possible ARTLENS visit outcomes. The research comprised

− discussions with CMA staff in the interpretation and digital innovation departments about the goals of ARTLENS Gallery, researching how other museums had attempted to tackle related topics and assessing the best ways to learn from visitors while meeting the specific objectives of the study;

− 36 interviews conducted with a mix of ARTLENS Gallery visitors (n=14) and non-ARTLENS Gallery visitors (n=22) (in October 2017);

Ten of the respondents were also first-time visitors to the CMA. In addition to answering a variety of open-ended interview questions, respondents were asked to examine some closed-ended questions that the team was piloting.

− second phase of this mixed-methods evaluation: a series of pre-visit and post-visit surveys followed by the final data analysis and reporting.

When visitors arrived at the museum, a random selection of adult visitors were asked if they would be willing to participate in a research study. For those who agreed the following steps were undertaken:

− participants completed an anonymous pre-visit survey on an iPad, which asked a series of motivational, perception, and demographic questions;

− upon completion of the survey, the respondents were given a numbered tag (this allowed the researchers to pair pre-visit and post-visit surveys while maintaining participants’ anonymity);

− visitors could then explore the museum however they preferred, with no guidance from the researcher; − after their visit (generally lasting approximately two hours), participants returned with their numbered tag and were given a post-

visit survey to complete. Many of the questions asked on the pre-visit survey appeared again on the post-visit survey in order to measure changes prompted by the museum visit. If the participants indicated on the survey that they visited ARTLENS Gallery during their time at the museum that day, then

they were given a series of additional questions about that experience. Since a central goal of ARTLENS Gallery is to be “a launch pad for exploration of the museum’s collection”, it was vital that evaluators can determine at what point of the visit ARTLENS Gallery was experienced. It was important to test the assumption that visiting ARTLENS Gallery at the beginning of a museum’s visit would create a more positive impact than experiencing the space toward the end of a visit. During November 2017 and January 2018, 438 paired surveys were collected. In order to ensure a substantial sample of target audience populations, particularly millennials and those visiting with children, additional surveys were collected from these groups. In total, approximately 36% of the respondents visited ARTLENS Gallery. Results The visitors who were most likely to opt into the ARTLENS Gallery experience were younger adults, families and non-members. Visitors who were motivated to visit the CMA that day by a desire to have fun and be entertained were also more likely to visit ARTLENS Gallery. If they visited ARTLENS Gallery early in their visit, they were also more likely than other visitors to go into the museum’s permanent collection galleries. When asked to reflect on their experience and what they found meaningful about it, participants were likely to agree that

− the visit to ARTLENS Gallery enhanced their overall museum experience (76%); − encouraged them to look closely at art and notice new things (74%); − increased their interest in the museum’s collection (73%); − increased their perception of the CMA as a place that was welcoming to a wide range of visitors (78%) and was forward-thinking

and innovative (76%). In the survey, participants were asked to use their own words to describe their experience at the CMA and, when applicable, in ARTLENS Gallery. Compared to the CMA, participants tended to describe ARTLENS Gallery using words that fell into the category of fun or entertainment (e.g., “fun, “enjoyable,” “play”); new (“modern,” “innovative,” “different”); or interactive (“interactive,” “engaging,” “lively”). Additionally,

− individuals who visited ARTLENS Gallery had more positive views about technology in museums at the outset but also had a statistically significant increase in their perceptions that was substantially higher than visitors who did not visit ARTLENS Gallery.

− one of the findings across all ARTLENS Gallery visitors showed the differences between their pre-visit and post-visit survey responses around art comprehension. Those who visited ARTLENS Gallery felt less confident about their level of art understanding and knowledge at the beginning of their visit compared to those who did not visit ARTLENS Gallery. After a single visit, ARTLENS Gallery visitors substantially increased their ratings, whereas visitors who did not go to ARTLENS Gallery did not experience a significant change. This was especially true for ARTLENS Gallery visitors who felt less confident in their understanding of art upon entering the museum that day.

Millennials and families The research showed little difference between frequent and infrequent visitor groups’ responses, however there were substantial differences among two particular visitor segments: millennials and families:

− millennials (adults born between 1981 and 1996)1 were more likely than older adult visitors to visit ARTLENS Gallery (44% compared to 29%);

− millennials who visited ARTLENS Gallery were most likely to visit museums as a way to have fun (37%) and relax (38%); − millennials’ art understanding and knowledge were more likely to increase significantly over the course of a single visit if they

attended ARTLENS Gallery, rising from a base rating of 5.3 to 6.0, compared to those who did not visit ARTLENS Gallery (5.8 to 6.0);

− ARTLENS Gallery positively influenced millennials’ perceptions of art museums as good places to have new experiences; their agreement with this item rose from 76% to 88% by the end of their visit.

As for the families, their part in ARTLENS Gallery constituted more than half of visitor groups. According to the study, “with its interactives focused on promoting group engagement, ARTLENS Gallery proved to be a natural match for families, who were primarily motivated by a desire to facilitate a positive experience for the entire group”. Thus, families who visited ARTLENS Gallery were more likely to leave fully satisfied with their CMA visit experience. Over a single visit, family group participants who visited ARTLENS Gallery also had a statistically significant change in their art understanding and interpretation skills compared to families who did not visit ARTLENS Gallery. Art Museum Digital Impact Evaluation Toolkit

The research brought also the Art Museum Digital Impact Evaluation Toolkit that “aims to bring [study’s] findings and methodologies out to expand this investigation into interactive technology outcomes throughout the art museum field”. The toolkit focuses on determining whether the set goals and associated metrics for measuring them (such as whether the interactive technology expands visitors’ art knowledge and understanding skills or whether it changes visitors’ perceptions of what an art museum experience can be) can be adopted in other environments.

Benefits The study provided new ways of capturing insights into how interactive technology affects the visitor experience. According to the research results, “capturing visitor attitudes, perceptions, and self-reported skill levels prior to and immediately after their visit gave a chance to determine real changes that developed”. Additionally, “asking layered questions also helped identify and parse out museum-based language barriers, improving researchers’ abilities to gather meaningful data more easily”.

Costs & Timeframe The research was held in the period of 2016-2018

Additional info & comments

ARTLENS Gallery’s’ 2019 awards: Communicating the Arts Grands Prix Awards Silver: Cross-Institution Partnership, Launching Open Access: Toolsets and Protocols for Best Practice Gold: Participatory Experience, ARTLENS Gallery Gold: Permanent Exhibition Design, ArtLens Exhibition Media Technology MUSE Awards Bronze: Research and Innovation, Open Access Initiative GLAMi Museums and the Web Awards Finalist: Behind the Scenes, Open Access Initiative

Involved Parties National Endowment for the Arts, Rockman et al

Contacts Cleveland Museum of Art: Jane Alexander Chief Digital Information Officer jalexander@clevelandart.org

Art Institute of Chicago

Who Art Institute Chicago, US What Data visualisation tool

Where Chicago, US When starting from 2014

Description The Art Institute Chicago has developed and implemented the data visualisation tool that helps to predict visitor attendance by means of applied data and analytics within the organisation. Andrew Simnick, Senior Vice President for Finance, Strategy and Operations at The Art Institute of Chicago shares the insights of implementation in “How the Art Institute Uses Data to Predict the Future” podcast, published in 2018.

Incentive The attendance modeling project started with the need to understand the basic factors that drive attendance to the Art Institute. The team aimed to understand how such factors as the effects of exhibitions, seasonality of weather and other impacted visitation. To path the way to data and analytics usage within the organisation, Andrew Simnick and team built a model that tested 200 factors, both internal and external, from exhibition type, to time of the year, weather, events in Chicago, Chicago public schools schedule, tourism, etc. Further, the factors were narrowed to 20 and prioritised based on two things:

− geography − channel.

The process was followed by institution’s first major effort in data visualisation and setting up a dashboard that was easy and intuitive for the end user to see when there is a meaningful change in attendance overall, by geography, by channel. The purpose of the dashboard was presentation of the mathematical modeling and regression analysis “in a way with not much resource investment for staff around the museum”. The Institute modeled by the week which gave “the best balance of a usable output while still having clean enough information to make good decisions”. The first full year of using the model for prediction allowed the institution to be within 1% of total attendance, ticketed revenue and per-capita revenue. Method The most voluminous and resource-consuming work is preparing data for analysis. Simnick stresses that the Art Institute early on encountered significant limitations in staff capacity to identify, maintain, and process data sets. This quickly became the main impediment to the opportunity of extending comprehensive analytic efforts across the museum. To overcome these constraints a “business intelligence core” was created for the museum - a combination of data warehousing and software. Automation has been introduced at different stages in the preparation process, dramatically reducing the time required to access and process internal data. This also allowed to incorporate external data sets, including demographic, psychographic, and geographic information, where appropriate. Investment in process optimization and in automation has allowed to increase institutional efforts with less need for staff or consultant time. According to Andrew Simnick, the first version of the modeling was done mainly manually. The data was extracted from the Art Institute’s ticketing systems and combined with the external data such as weather, or mapping zip code to geography. The team had to do some internal build out such as data input platforms to get the information needed for the modeling. The platforms were built using in-house systems, without much financial investment. Factors for modelling

− tourist inflow in the city (Art Institute chose to purchase hotel data and feed it into the model);

− seasonality (the team observed spikes in attendance around the few days after Thanksgiving to the last week in December between Christmas and New Year);

− weather (a slightly unfavourable weather still in line with the season is the most favourable for attendance); − ticket pricing; − type of the exhibition.

According to Simnick, not all exhibitions are doing the same in terms of attendance but all of them have to be in the model. The system leans on the data aggregated within 15 previous years Data visualisation According to Simnick, one of Art Institute’s first analyses using visualisation highlighted different drivers of visitor attendance. Generally, the Institute’s attendance was difficult to interpret, outside of generally higher attendance in summer than in winter. However, looking at attendance patterns across different purchasing channels and geography, the insights have surfaced. For example, US tourist volume from fly markets (i.e. 8 or more hours from Chicago) follows a seasonal pattern and is only marginally dependent on exhibitions. The members attendance , in its turn, is almost flat outside of major exhibitions. Acknowledging these insights havs changed the Institute’s strategies on when and how to approach different segments of audiences. Leaning on attendance patterns across different purchasing channels and geography, the Institute created a series of models to determine which internal and external factors matter by channel. The institution designed a visual dashboard to identify and display when meaningful changes in attendance occur.

The Art Institute implemented interactive visuals in other areas of the museum to translate insight to operational decision-making. For example, with the help of interactive visuals, the Art Institute now has 99%+ of its on-display objects online with an image. The institution also used interactive visuals to help reduce complexity and SKU count within the museum shop while maintaining projected revenue. On-site gathered data

− measuring Wi-Fi signal to estimate the percent of visitation overall

The data from the museum’s Wi-Fi network also allows to analyse visitors’ routes and dwell time throughout the galleries. It showed in particular that visitors follow a similar path through the museum regardless of where they enter. This approach has also revealed that using exhibition-like interpretive techniques in the galleries which display the institution’s permanent collection significantly increases dwell time there. While this data is less precise than direct observation in individual galleries, it allowed the institution to quickly observe meaningful trends and make fast and informed decisions.

This allows the institution to experiment with different types of programs as well as to understand which galleries are getting the most traffic. This is not measured at the individual level but rather through broad trends based on the Wi-Fi signal. One of the further findings which influenced the museum’s approach to exhibition programming is that small permanent collection installations can significantly change the visitation patterns which are typically seen in the museum. An example brought by Andrew Simnick is a permanent collection show called “Flesh” images from which (depicting Ivan Albright, who has deep ties to Chicago) generated a high response on social media. What the team did, is a significant push on social media around the images in the show. The outcomes were noticed during the run of the show: the online response was incredibly favorable and higher than normal for a permanent collection rotation, while the dwell time in the gallery itself went up by 25%.

− volunteers with clipboards, stopwatches, intercept interviews. The institution also relied on the staff and volunteers to help build data sets when a quick answer was needed. For example, the Art Institute worked with volunteers to capture over ten thousand visitor questions over eight weeks, revealing information about museum’s most popular works and gaps in the frontline service. According to the institution, this approach helped to make informed decisions quickly and save some time answering complex questions. Outcomes/updates According to Chicago Business, as a result of using the predicting model, the Art Institute focused on smaller exhibitions, opening a new show, on average, every two weeks with the aim to boost the attendance. In 2018 it hovered around 1,6 million a year, with members visiting once every nine to 12 months and local ticketed visitors once every 18 to 24 months. The expectations behind the decision was reaching the 60% annual renewal rate for institution’s 100,000 members.

Benefits According to Andrew Simnick, in the first year of using this tool, the institution made decisions surrounding marketing spending and content that resulted in a gain of over $2 million in net revenue from admissions.

− creating a predictive attendance model with <1% error rate by identifying internal and external factors that matter most;

− understanding how visitors use the physical space by applying data from existing systems in new ways.

Contacts Art Institute Chicago: Andrew Simnick Senior Vice President for Finance, Strategy, and Operations asimnick@artic.edu

HERIT-DATA

Who HERIT-DATA What Visitor flow tracking system

Where Project leader Tuscany Region (Italy) When ongoing

Description HERIT-DATA is a project that aims to reduce the impact of tourism activities on cultural heritage sites and to do it through innovative solutions developed with the support of new technologies and data solutions. The project focuses on sites of particular archaeological, historical and cultural interest, including UNESCO World Heritage Sites. HERIT-DATA partners work together to develop, test and transfer a series of tools to collect, generate, integrate, analyse information and transform it into behaviour changes. The results will also contribute to improving decision-making processes in public administration and tourism or heritage managing bodies. Project’s scope: 7 countries, 48 months, 4.2 million, 12 partners. Project leader: Tuscany Region (Italy) Project partners: City of Dubrovnik development agency (Croatia), Turisme Comunitat Valenciana (Spain), AVITEM (France), Center for Spatial Research (Bosnia and Herzegovina), Conference of peripheral maritime regions of Europe (France), Santa María Real Foundation for Historical Heritage (Spain), Valenciaport Foundation (Spain), Foundation for Research and Innovation (Italy), Region of Western Greece (Greece) Occitanie Region (France), Faculty of Sciences and Technology (Portugal). The project will be implemented in six pilot sites in Dubrovnik (Croatia), Pont du Gard (Occitanie Region, France), Ancient Olympia (Western Greece Region, Greece), Florence (Tuscany Region, Italy), Valencia (Valencia Region, Spain) and Mostar (Bosnia Herzegovina). The expected outcomes of HERIT-DATA are the following:

− collecting and generating data; − developing a platform for processing and exploiting the data collected in the pilot sites; − building and testing an app to better organise tourist flows; − providing useful data to policy makers to elaborate new tourism public policies; − providing useful insight to tourism and heritage managers.

Pilot sites: first sensors installed to measure the flow of visitors in Valencia. In September 2019 E-flash edition, it was explained how HERIT DATA’s partners Valenciaport Foundation and Fundación Santa Maria La Real will install sensors to collect data about tourism flows in key areas of the city. The setup of these sensors is ongoing: after the first three sensors in the port terminal, two others will be installed at the Basilica of the Virgen de los Desamparados and at a crossing point in the city centre. A total of 11 sensors are planned to be installed with the support of the HERIT-DATA project and other projects led by the municipality. The sensors are aimed at capturing the number of mobile phones with the Wi-Fi connection activated, but without collecting personal data in order to preserve privacy. In addition, Fundación Santa Maria La Real is going to install other data collection technologies that will analyse the temperature, humidity and lighting of the Basílica de la Virgen and the Cripta de San Vicente. This will allow to look at how the presence of big groups of visitors every day can impact the historical and cultural sites. All this data will converge in the HERIT-DATA platform which is designed to improve decisions related to the management of the tourism offer in Valencia and in the other pilot sites. A web-app for tourists will also be developed and integrated to some existing tourist apps that are currently used by visitors. The aim is to provide tourists with the real-time information about the crowdedness of some of the sites they visit.

The choice of Valencia as a project’s pilot is connected to the issue of overtourism the city is experiencing. According to HERIT-DATA’s information, between 2006 and 2018, the number of tourists per year has increased from about 1,6 million to 2 million. Cruise traffic at the Port of Valencia has grown 125% in number of passengers over the last 10 years. Like in many other Spanish and European cities, this increase of tourist flows has also had negative impacts, such as the overcrowding of the city centre or the increase of the rental prices (only between 2017 and 2018, the increase in Ciutat Vella is of 29,53%, according to ASICVAL, Asociación de Inmobiliarias de la Comunitat Valenciana). HERIT-DATA’s partner for data collection and visualisation HERIT-DATA cooperates closely with the Snap4City platform which offers smart city solutions capable of bringing valuable data from different areas of the city life. The access to such platform can be highly instrumental for museums in tracking and identifying potential visitors and in being aware of their routes, preferences and behaviour. Snap4City is a platform providing a flexible method and solution to create a large range of smart city applications using heterogeneous data, performing data analytics, and enabling services for stakeholders by IOT/IOE, data analytics and big data technologies. Snap4City applications allow to use multiple paradigms as data driven, to stream and batch processing and thus to be instrumental for the following agents:

− Smart Living Lab users and developers (providing solutions to develop applications without vendor lock-in and technology lock-in); − final users of customisable/flexible mobile apps and tools; − city operators and decision makers using city dashboards and IOT/IOE applications for city status monitoring, control and decision

support; − organisations/communities interested in Smart City and IOT “which would like to perform experiments on smart city data, that can

upload or reuse from those available, for research purpose and validation”.

Features offered to the organisations: − access to a large collection of data coming from different cities; − exploiting a set of tools for uploading and integrating new data and performing data analytics; − comparing results of different cities; − sharing data transformation and data analytics with other users in the same or different organisations; − access to a large set of training test cases, tutorials, videos and examples; − searching and discovering smart city data on the basis of entity relationships, temporal and spatial, semantic search; − access to Advanced Smart city API, also in the form of MicroServices in Node-RED; − control of entity type access with GDPR compliant mechanisms; − uploading new data sets up to 30Gbyte; − authorising a number of final users for the organisation.

Snap4 technology is used for developing integrated sentient solutions in the domains of smart city, Industry 4.0, Smart Home, Smart Farm, Smart Health, smart retail, etc. Snap4City is capable to keep under control the real-time city evolution through reading sensors, computing and controlling key performance indicators, detecting unexpected evolutions, performing analytics, taking actions on strategies and alarms. The platform for city stakeholders Snap4City in action: People Flow Analysis via Wi-Fi Snap4City performed the data analysis via Wi-Fi Access Points in the city of Florence with the aim to evaluate tourists’ behaviour and thus to better understand the movements in the city. In this particular case, Wi-Fi Access Points were used as sensors to capture and understand city users’ behaviour with a significant precision rate (precision achieved with heatmaps, origin destination matrices, trajectories and predicting user density). The raw data was processed by a set of data analytics algorithms to compute:

− permanence time of tourists in the city, balance of citizens and tourists; − preferred trajectories and locations; − OD matrices for each time slot, inbouds and outbounds; − amount of new and returning visitors: Regency and Frequency analysis.

From the analysis of the OD matrices and/or OD Spider Flows, for example, it became evident that different parts of the city were used diversely by different city users. AP presented various trends in the usage of the Wi-Fi along the 24-hour period and in the different days of the week. Generally, the data collected from the Wi-Fi network in Florence helped to identify the averaged trend along the day, for each AP, for each day of the week. This resulted in 345 APs, on 7 days, 24 hours per day. Classification of City Areas for User Behavior The following chart shows user behaviour patterns (marked by pins of the same colour) corresponding to the certain area. It can be noticed that there are areas which are very active in the morning time whereas some clusters provide an evident activity in the afternoon. A few areas present significant activity in the late afternoon and in the first hours of the night.

City usage predictions and anomaly detection On the basis of the clustering and historical values, for each AP, the predictions on the number of people attending a certain area of the city were computed, and thus probable movements and needs were identified. The approach was used for anomaly detection and early warning when the actual data differs from the predicted. The solution was developed by DISIT lab with the joint effort from the side of the city of Florence in the Wi-Fi network instrumentation. DISIT lab contributed with the algorithms, tools and dashboards. The platform for the city users Toscana Where What … Km4City, Toscana in a Snap apps Toscana Where What .. Km4City and Toscana in a Snap are multipurpose mobile and web apps. They cover multiple domains and offer a range of smart features for the city users. Among such features, providing real-time information from certain areas of the city.

Mobility and transport: − routing (car, bus, bike and pedestrian) and multimodal routing; − smart parking, bike sharing; − timetables of public transport operators: bus, railway, ferry, tram; − bus tickets in the region, − navigator and connected drive; − vehicle monitoring, via OBD2 data collection, and storage on cloud; − car position save and recover and more.

Environment and weather: − pollution and pollutant, heatmap and subscription to be notified when critical conditions are detected; − sensors values for pollution, pollination, etc.; − heatmaps for environmental, weather and more.

Social: − recent Tweets; − discussion forums; − receiving engagements with the city operators, etc.;

Such mobile apps provide a usable data analytics with data collected and made accessible on dashboards. The users may use the app anonymously or by registration providing or not a signed consent according to GDPR. The most relevant data analytics collected from the app are related to the: preferred places of the users, origin destination matrices, trajectories, user behaviour, comments on services, images on services and new POI, ranking and appreciations, etc. With the above insights, city agents can get a real time view of the city and regional status of traffic, mobility, parking, bike sharing, triage, event of traffic, public transportation, etc. As Snap4city explains, overall, “the app is a tool for the users for collecting their personal data which is located at their exclusive disposal on the Snap4City platform according to GDPR. They can save trajectories, personal usage, etc., and exploit this data with IOT data of the city for creating IOT Applications and personal dashboards. This allows to create a participatory community and a group of active city users which can contribute to the day by day activity of the Living Lab”. The above app solution was implemented in Tuscany region, Antwerp and Helsinki.

Benefits − joint cross-institutional efforts in tourist flow optimisation; − a popularisation of the unified platform for cities allowing to gather and visualise immense amount of data from different sectors.

Additional info & comments

The Covid-19 pandemic has brought many unforeseen challenges in the tourism sector, with no exception for HERIT-DATA’s pilot sites in Dubrovnik (Croatia), Pont du Gard (France), Ancient Olympia (Greece), Florence (Italy), Valencia (Spain) and Mostar (Bosnia Herzegovina). All the sites had to shut down for several weeks due to lockdown measures, and their re-opening did not bring the same numbers of visitors than in previous years. Most of them had to count on domestic visitors rather than international ones.

Involved Parties HERIT-DATA is an initiative of Interreg Mediterranean. HERIT-DATA is a part of the Interreg MED Programme 2014-2020 within which partner states from 13 countries are working together in the transnational European Cooperation Programme for the Mediterranean area. The transnational setup “allows partners to tackle challenges beyond national borders, such as the rise of low carbon economy, the protection of natural and cultural resources and the strengthening of innovation”. The main objective of the Interreg MED Programme is to promote sustainable growth in the Mediterranean area by applying innovative concepts and practices and by supporting social participation “through an integrated and territorially based cooperation approach”.

Contacts HERIT-DATA: Chiara Guiggiani The Foundation for Research and Innovation, University of Florence chiara.guiggiani@unifi.it

Whitney Museum and Google Analytics

Who The Whitney Museum (US) What Google Analytics

Where New York City, US When 2018 and ongoing

Description In the article “Dimensions of Museum Data”, Colin Brooks, Senior Developer at the Whitney Museum describes the museum’s experience of working with Google Analytics and the ways in which the team optimised this tool for more efficient results and lesser labour load. Brooks highlights that Google Analytics is not easily adaptable for cultural institutions’ purposes since “it was not built for the arts”. And while the standard metrics such as pageviews, session behavior, referrals, ecommerce still matter to the museum, when it comes to the cohesive exhibition data analysis, “it’s become more and more labor intensive to identify the throughlines between content spread across different aspects of the online presence”. To tackle this challenge, the Whitney Museum is optimising the utilisation of custom dimensions. Collin Brooks further explains that custom dimensions allow to create an institution’s own organising principle for its data and thus to associate behavior by internally important concepts like exhibitions, events, artists, or visitor motivations. Dimensions can slice across all the metrics which is tracked, but provide different ways to frame the investigations of the content. Challenges that the work with Google Analytics, according to Brooks, brings:

− “adding more tags and the burden (both human and technical) of more platforms fragments institution’s ability to cohesively analyse its data”;

− the above impacts institution’s ability to answer basic questions about its operation; − the need of “faster ways to look at the data in the proper contexts instead of checking and supporting a large number of reporting

platforms”.

Main question How can the institution measure the performance of online exhibition content before and after it opens? In this regard, two factors are essential: time and performance relative to that time. Web analytics data can potentially tackle these two aspects showing time when exhibitions are open and closed and providing all other metrics in regard to performance. But the challenge which Brooks outlines is how much labor would be involved in comparing data across exhibitions. Problem Since exhibitions do not have same schedules, something as basic as choosing the time range in Google Analytics means doing separate reporting for every show, since Google Analytics has no concept of an exhibition opening date. In case with the Whitney Museum, the second problematising factor is that not all of institution’s exhibition focused content is located on a single page. Sections like essays, or audio guides, or Whitney Stories are located outside the exhibition pages, so any review centered only on whitney.org/exhibitions/programmed, will not cover everything the institution needs it to. In addition, the related behavior data is tied to a number of other exhibition performance sections like tickets or memberships online or related resources like the collection or events. Brooks concludes that “exhibition performance depends on multiple aspects of whitney.org, and a concept of time that is not natively accessible within Google Analytics”. Solving the problem Based on the assumption that online visitors interested in an exhibition will visit the exhibition page on whitney.org, the Whitney Museum’s team created a number of custom dimensions helping to track the exhibition performance across the website. The dimensions which were added for exhibitions all depended on a user’s first visit of the exhibition page. Once users did it, their session was tagged with three dimensions:

1. Exhibition. The title of the exhibition. 2. Exhibition phase. “Present,” “future,” or “past” depending on if the exhibition is currently open, yet to open, or closed. 3. Exhibition relative date. A numerical value representing the number of days from the opening date (4 days before would be “-4”,

10 days after would be “10”, and opening day would be “0”). These three dimensions allowed the Whitney’s team to build reports based on a single exhibition reviewing data from everything users did across the site during their session which could comprise watching videos, browsing collection, reading about a show, purchasing tickets, etc. They also allowed to build reports for all exhibitions, but limit them to a specific phase (e.g., pre-opening), or an exact range of days (e.g., the week before opening). And generally, this attribution to Google Analytics structure made the process of data collection less labor-intensive. Other benefits:

− being able to easily attribute the ticket and membership sales to specific exhibitions; − offering up easier and more human-friendly ways to view new data (instead of manually checking the dates for a specific exhibition

it became possible to create reports framed in ways that cover many).

Exhibition data gathered with the given approach after 9 months since the start of 2018: − 66% of exhibition traffic has been to exhibitions while they were open, 19% to ones that were closed and 15% to ones opening in

the future; − users viewing presently open exhibitions spent 20% longer on the site on average than those looking at past or upcoming ones; − 83% of exhibition-driven revenue came from shows that were currently open and 16% from ones opening in the future; − 76% of exhibition-associated audio guides views came from shows that were currently open and 20% from ones that have already

closed.

Limitations − most users are visiting only one exhibition page; − the last exhibition users view is the most important one for the purposes of museum’s data (as that last-view will override any

earlier exhibition-related values for their session); − aside from the above three exhibition-focused dimensions, Whitney has 9 more that cover other aspects of the site. Three are used

in the same way as exhibitions for events and the remaining 6 are used for things like tracking audio guide and Mobile Guide usage, language settings and visitor motivations.

Further usage of Google Analytics in Whitney In her article, “Visitor Motivation Survey and Audience Segmentation for the Whitney Museum of Art Website”, Sydney Stewart gives a summary of the pilot research project at Pratt Institute undertaken by her and her colleague Samantha Nullman in collaboration with the Digital Media department at the Whitney Museum. As a response to “increased visitation and interest to the Whitney Museum’s website” and with the purpose to “better understand how their digital visitors are interacting with it”, in 2018, the institution and the Pratt Institute launched a visitor motivation survey (VMS) that asked users why they came to the whitney.org at the specific time. The research was aimed at “helping the Whitney Museum to better understand the website users by creating user segments based on the motivation survey and Google Analytics”. Methodology The tools chosen for the research were a single question survey on the Whitney.org combined with Google Analytics which provided insights on website interactions and allowed to segment web visitors by why they came to the website. The first stage of the research comprised the visitor motivation survey (VMS) being available for two weeks (March 6 — March 20, 2018) on the museum’s website. During this time the survey accumulated 11,878 responses.

The survey offered the website visitors one single question: “Why did you come to whitney.org today?” and comprised the response variants such as “I enjoy experiencing art”, “I am preparing for my visit”, etc. In Google Analytics, an event was created for the Visitor Motivation Survey every time a user selected a response and each response was recorded in the event label. All the selected responses made the rest of users’ interactions tracked under the segmentation specific to the chosen response. Google Analytics reflected the survey responses with the following segments (and respectively assigned motivations):

− New Visitor; − Returning Visitor; − Explorer; − Professional Researcher; − Learner; − Artist; − Opportunity seeker.

Google Data Studio (a free data visualization tool that connects to data sources, including Google Analytics and creates interactive dashboards) became a further tool to create dashboards of key metrics and thus to visualise how the separate segmentations interacted with the website during their visit. Pratt Institute and the Whitney Museum prepared the visualisation where the first page contained basic demographic information such as age, gender, location and devices used to access the site and the second page focused specifically on behavior or content interactions. In her case overview, Sydney Stewart shares examples of Google Data Studio pages for Overall Users.

According to the research overview, using Google Data Studio “made it quick and easy to distinguish significant similarities and differences across the segments and define common behavioral trends” and “highlighted specific areas that would be useful to do more in-depth analysis and made pulling the data easy”. Some of the research findings:

− split between visit-related and collection/content-related visits (New and Returning visitors made up half of the segment sessions, while content driven users (Researcher, Enjoys Art, Interested in Contemporary & American Art, and Interested in Engaging) made up the other half. The split varies depending on certain factors influencing website traffic, such as academic year, exhibitions, or programming);

− audio and video content reach (Interested in Contemporary Art and Researcher users were the most engaged with the audio content; the percent of each segment that engaged with video was often higher than the percent of each segment that engaged with audio);

− object and exhibition page reach (users looking to explore more about art and the collection spent more time and clicked through mostly exhibition pages; Enjoys art and Return Visitors users viewed more exhibition pages than other segments; New Visitors were less interested in exhibition pages; Researchers were the most engaged with object pages compared to all the other segments). According to Sydney Stewart, using Google Analytics for this research project played a major role: “The pairing of the Visitor Motivation Survey with Google Analytics was the first learning opportunity. Google Analytics can tell a lot about user behavior but not why they came to the site and their personal interests. Alternatively, the Visitor Motivation Survey, without Google Analytics, only provides the basic segments of who the users are but not what they do on the site. Combining the two allowed for greater insights into user behavior overall.” Advantages of using Google Analytics and Google Data Studio As Sydney Stewart reveals, that “the initial challenge was the need to define what metrics would be most useful to focus on because it is very easy to start drowning in all the data that Google Analytics can provide” but “after setting the key demographic and behavioral metrics, it became clearer where to go with the direction of dashboards”. As for the usage of Google Data Studio, it significantly facilitated the work with the data for all seven visitor segments since it gave the opportunity to copy each dashboard and switch the segment filter and consequently allowed the desired data to be visualised consistently across every segment. Stewart concludes that “despite some limitations of

the tools, the combination of a Visitor Motivation Survey, Google Analytics, and Google Data Studio allow for a useful method of web user evaluation and data visualization that can easily be replicated for repeat studies and compared over time”. According to the author the Whitney continued the research proceeding with a second VMS survey.

Benefits − a well-established tool adjustable for the institution’s tracking needs; − utility for further research endeavours (such as visitor segmentation).

Drawbacks − technical difficulties with adjusting Google Analytics to the mode of cultural institution’s operation

Involved Parties Google (Google Analytics, Google Data Studio)

Pratt Institute

Contacts The Whitney Museum: Colin Brooks Senior Developer colin_brooks@whitney.org

Shrewsbury Museum and Art Gallery

Who Shrewsbury Museum & Art Gallery (SMAG)

What “Intelligent Exhibit”, adaptable sound descriptions

Where Shrewsbury, UK When 2017

Description In 2017, Microsoft partnered with Black Radley and the Shrewsbury Museum & Art Gallery to create a solution that would react to museum visitors as they interact with exhibits based upon their approximate age, gender, and emotional state. Additionally, the solution was meant to provide detailed insight on how visitors navigate through the museum and which exhibits they spend more time with. The aim of the project was to assist the museum in attracting the visitors and better understanding how they use the museum. Shrewsbury Museum & Art Gallery offers a view on 650 million years of Shropshire's rich history comprising five galleries: Roman, Medieval, Tudor, Stuart and Shropshire Gallery. It is located in the 19th century Music Hall and 13th century Vaughan's Mansion. Shrewsbury Museum & Art Gallery is owned and operated by Shropshire Council. Challenge of the project According to Black Radley, the company behind the project, provincial museums like SMAG are struggling to be more attractive for the visitors and to ultimately drive higher attendance from the general public. Thus the developed solutions were aimed at:

− making exhibits more attractive by using cognitive intelligence to give a tailored experience based upon demographic information; − tracking and reporting on how visitors interact with exhibits and the museum as a whole to allow the museum to better understand

their audiences.

Process of solutions’ implementation − installation of the Visual Studio Code; − obtaining the key for the Cognitive Services Face API. − obtaining an Azure subscription to use Azure App Service and Azure Table storage; − setting up Windows 10 IoT Core on your Raspberry Pi 3. − setting up an account for Power BI.

The developing team was aimed at creating a device that would enable visitors to approach an exhibit in the museum and hear an audio that was specially tailored for this specific demographic group. Such customisation was meant to provide a more enjoyable, entertaining, and informative experience. The realisation of such “Intelligent Exhibit” presupposed placing an IoT device with a webcam and a speaker on each exhibit. This device ensured the following:

− detecting the presence of faces gazing at or interacting with the exhibit; − greet visitors appropriately; − taking a photo and obtaining the rough age, gender, and emotional state of each face by using Face API; − playing a suitable audio description to match the age and gender of the visitor (for example, when looking at an exhibit that is a

panoramic painting of Shrewsbury, a visitor in the 12–17-year-old demographic heard an audio that was pacey and enthusiastic, and the script included age-appropriate cultural references to an iPhone while the representatives of 55–64-year-old group heard an audio that was more formally delivered and containing more specific details);

− detecting visitors’ departure and pausing the audio description. The IoT device was a Raspberry Pi 3 running Windows IoT Core, which runs a UWP app that does the initial face detection, takes the photo, and plays the audio descriptions. The Raspberry Pi is equipped with a webcam and a speaker, but no screen or other peripherals.

Visitor data and its analysis

When the Intelligent Exhibit device captured the details of a visitor’s face, Microsoft stored the unique face ID and added it to a list of faces for that day. The face data was stored with the time, exhibit location and device details so that it was possible to determine whether a face had been seen before on that day, and if so, at which exhibit and at which time. The solution logged each sighting so that the data could be analysed at a later date. Using the above approach, key data about each visitor could be collected:

− approximate age, gender, and emotional state for each visitor at each sighting; − which exhibits a visitor visited and in which order; − how much time a visitor spent at each exhibit.

Technical delivery

According to Black Radley, the key steps of the solution were to create a UWP app that would run on an RP3 device running Windows IoT. This device would be integrated into exhibits and set to capture images when a face is detected. The photo would then be sent to a proxy Web API that passes the image through the Face API to gather data on emotion, age, gender, and other data points before logging the data in Table storage for later analysis and passing the data back to the UWP app. The UWP app would then tailor the audio description about the exhibit based on the data gained from the API.

Power BI for data analysis The immense volume of the collected data needed to be efficiently visualised to be meaningful and usable for the museum. It was discovered that very quickly and without data conversion from raw table storage, the data stored in Azure Table storage could be used to produce a report using Power BI Desktop. To create the report, the data had first to be imported using Get Data. After selecting Azure, the following step was selecting Azure Table Storage. And after the data source was configured, it was then just a case of dragging visualisations onto the report canvas to visualise the data with a dashboard.

Outcomes The solution achieved the primary goals of making museum exhibits more attractive for visitors and collecting visitor data. The approach involved a wide range of Microsoft technologies but centered around three main areas:

− The Universal Windows Platform to capture visitors via an IoT device; − Cognitive Services to provide intelligence about visitors as they interacted with exhibits; − Azure App Service for storage, hosting, and analysis.

Next steps for the solution offered by Black Radley

− to compare how visitors interact with exhibits with and without narration (gathered information can be used for visitors segmentation based on their preferred exhibits and their dwell times);

− to create an easy-to-use interface for making modifications to narrations (the solution requires some technical expertise to configure the device and to update the narration);

− to adapt the software to allow for different narrations for various devices (the initial solution was constructed as a single stand-alone device, however, the low cost of the device makes it possible to have multiple devices, which either interact with or track visitors through a museum or gallery);

− to allow for museum professionals to share performance information easily with other museums and galleries.

Benefits − visitor data collection solutions fully integrated in the exhibition and non-disruptive from the standpoint of the visitor journey; − the solution is entirely open source on GitHub at Intelligent Museum Exhibits (Patron Interactive Engagement); − enhancement of audio and visual technology by a customised narration; − narration conveniently adjustable to visitor feedback.

Costs & Timeframe According to Black Radley, the prototype was produced during a one week while availability of cheap and fast facial recognition software allows this to be done for under £100 using free (Open Source) software.

Additional info & comments

Full case study by Microsoft

Involved Parties Microsoft

Black Radley (a consultancy organization that works with public services)

Contacts Shrewsbury Museum & Art Gallery (SMAG): shrewsburymuseum@shropshire.gov.uk