practical data* visualization with javascript

1

PRACTICAL DATA* VISUALIZATION WITHJAVASCRIPT

WHEN, *WHAT, AND HOW

ceilyn_boyd@harvard.edu Program Management @ the Harvard Library

Ceilyn Boyd

2

ORIGINAL PRÉCISThis talk introduces a methodology intended to help you

decide:

When to use JavaScript to visualize your data (and when tochoose alternate means)What data, or aspects of your data to render visually, andHow best to use the tools at your disposal to visualize yourdata

3

WHYBackground and Motivation

4

SURVEYING THE LANDSCAPE

Image source.

Demand for, and popularity of data analysis, data mining,data-driven visualization has led to ...

5

...THIS

Image source.

6

SO MANY OPTIONSProliferation of technologies, languages, platforms,applications, and tools with overlapping functionality

- - - - - - - - - -

- - -

Proliferation of graphics and visualization libraries & APIs - - - -

- - , - -

Excel (PowerPivot, Pivot Charts) Google Sheets/ChartsJupyter MATLABOctave OpenRe ne Plot.ly R SAS Stata Tableau

JavaScript Python RubyWebGL SVG

Bokeh Chart.js D3.js Dygraphs.js Google Charts APIjqPlot.js Processing.org Processing.js VisPy Vis.js

7

SO MUCH DATALibrary-related data:

- - Harvard Libraryoperational data - -Program/project assessment data & results

Library-related APIs: - - -

Research data & APIs: - - -

-

ARL data Google AnalyticsHarvard Library Open Metadata

Dataverse API DPLA API HathiTrust WorldCat APILibrary Cloud

Census.gov Factiva API New York Times APIsSEC.gov Twitter API

8

MANY QUESTIONS & CONSIDERATIONSWhen does visualization work best for libraries and librarydata?Data

How to model, structure, and manipulate data forvisualizationHow to work with data from different sourcesHow to decide when to use which visualization basedupon data

What makes a successful visualization?Concerns about knowledge, skills, and access totechnology and training

9

WE HAVE DATA. WE WANT TO ANALYZE &VISUALIZE IT...WHAT NEXT?

Image source.

10

PRÉCIS REDUX & REWRITTENThis talk will focus on a methodology intended to help you:

Choose amongst potential tools (including JavaScript) tovisualize your data

It will use three Library-related examples to:

Demonstrate the methodology in action, andDemonstrate how to use JavaScript for visualizationwork ows

11

AUDIENCEPeople who:

Want to add interactive visualizations to their Webdocuments (to create Data-Driven Web Documents)Are new or relatively new to programming but who arefamiliar with HTML, CSS, and some JavaScriptFor whom programming is not a frequent part of their dailyworkHave experience with programming but who do not usuallywork with data, graphics, or visualizationNeed to decide how best to create sustainablevisualizations

12

DATA VISUALIZATION WORKFLOWMETHODOLOGY & FRAMEWORK

Image source.

13

Goal: Reporting, Storytelling

DATA VISUALIZATION WORKFLOW

14

DATA VISUALIZATION WORKFLOWCOMPONENTS

15

WORKFLOW COMPONENTSDATA SOURCE

A component that provides data to the work ow. It could bea repository for any kind of data, such as a database, a le, an

API, a program; or it could be a function. Multiple datasources might be used in a visualization work ow.

16

WORKFLOW COMPONENTSDATA TRANSFORMER

A component that performs some kind of data processing,such as a lter or a le format converter. Multiple data

transformers may be needed in a work ow to prepare datafor visualizers.

17

WORKFLOW COMPONENTSDATA VISUALIZER

A component that presents a graphical representation ofinformation or data that is suitable for display on a device(e.g. via a Web browser or app), in a document, or in print.

18

DATA EXPLORATION WORKFLOW

19

APPLICABLE TO USE-CASES, SYSTEMS, TOOLS, &LIBRARIES

20

COMPARE DATA ANALYSIS & VISUALIZATIONOPTIONS

(See handout)

21

WORKFLOW CONSIDERATIONSWHAT TO CONSIDER WHEN BUILDING A VISUALIZATION

22

IMPLEMENTATION CONSIDERATIONS ARE...Related to an individual work ow componentRelated to implementation of a work ow segmentRelated to the entire work ow

23

IMPLEMENTATION CONSIDERATION CATEGORIESINCLUDE...Legal & EthicalTechnicalUsabilitySustainabilityAesthetic

24

COMPONENT CONSIDERATIONSComponent considerations involve:

DataComponent inputs & outputsRepresentation of the data within the componentPresentation of the data

25

SEGMENT & FULL WORKFLOW CONSIDERATIONSSegment/Work ow considerations involve:

Resources: e.g. Skills & knowledgeLicenses & agreemenets affecting use, sharing,presentation of resultsTechnical: e.g. compute capacity, bandwidth, security,scalabilitySustainability, reuse, automation,preservation/reproduction of results

26

SUMMARY OF COMPONENT & WORKFLOWCONSIDERATIONS

(See handout)

27

EVALUATING VISUALIZATIONSTOWARDS A STRATEGY

Network Graph 'Hair Ball', Source.

28

VISUALIZATION EVALUATION CRITERIA1. Data overview

The capacity of a visualizer to provide a global view of allsigni cant data elements simultaneously

2. Data navigationThe ease with which a user can use the visualizer tolocate/engage with a speci c data element

3. Expressive powerThe capacity of a visualizer to communicate the fullrange of relationships within the dataset

4. AestheticsThe visual utility and intuitive value associated with thevisualizer

Source: , Boyd & Wade, 2012.Visualizing Back-of-Book Indexes

29

IMPLEMENTING VISUALIZATION WORKFLOWSWITH JAVASCRIPT

30

BENEFITS OF JAVASCRIPTEstablished Web technology

Ubiquitous browser supportEasy integration with UI elements for buildingapplications (e.g. jQuery)Variety of frameworks for building applicationsMany visualization and data manipulation libraries

Standalone console & server-side support via Node.jsMany data manipulation librariesMany APIs/libraries for connecting to data sources (e.g.MySQL, MS-Access, Library Cloud, Dataverse)

31

BENEFITS OF JAVASCRIPT (CON'T)One language for data acquisition, analysis, visualization,application development, and document deliveryEnd-to-end visualization work ow supportNatural t for Data-Driven DocumentsUseful for prototyping and productionNo licensing fees

32

SUMMARY OF JAVASCRIPT CONSIDERATIONS

(See handout)

33

EXAMPLESJAVASCRIPT FOR DATA-DRIVEN DOCUMENTS

34

THREE HARVARD LIBRARY-RELATED EXAMPLES1. Selected Datasets for the Towards a Collections & Content

Development Strategic Plan Project (TCCDS) (HTML & CSS, with JavaScript & Node.js)

2. Selected Datasets for the Colonial North American Project (HTML & CSS, with JavaScript & Node.js)

3. OASIS Timeline Viewer Application (Data exploration: Browser only)

35

TOWARDS A COLLECTIONS & CONTENTDEVELOPMENT STRATEGIC PLAN DATASETS

DEMO

36

TCCDS WORKFLOW & CONSIDERATIONSWork ow

Considerations

37

COLONIAL NORTH AMERICAN DATASETSDEMO

CNA WORKFLOW & CONSIDERATIONSWork ow

Considerations

3839

OASIS TIMELINE VIEWERDEMO

OASIS VIEWER WORKFLOW & CONSIDERATIONSWork ow

Considerations

4041

DATA-DRIVEN DOCUMENT TEMPLATE/** * Create the drivers bubble chart of number of mentions * * @param String String containing JSON/TSV data * @return Void */ function createVisualizationFunction(dataString) // parse the string and create JSON objects var data = JSON.parse(dataString); // // process the data to get it into the proper // format for the target visualizer // var processedData = processData(data); // create the visualizer, initialized with processedData, adding it to a DOM element var visualizer = createVisualizerUsingLibraryFunction('visualization'

42

QUESTIONS?

43

THE ENDThanks to Bobbi, Reinhart, and Gloria for the invitation, and...

Thanks to you all for participating!

44

RESOURCES

45

WEB AND JAVASCRIPT RESOURCESGeneral

. Includes online references for HTML, CSS, JavaScript(client-side), Web APIs, Graphics (SVG, WebGL),MathML

JavaScript

. John Resig, BearBibeault

. David Flanagan. Bibeault, Katz

Mozilla Web technology for developers

Mozilla JavaScript Language ReferenceJavaScript Design PatternsSecrets of the JavaScript Ninja

JavaScript: the De nitive GuidejQuery in Action

46

VISUALIZATION RESOURCESVisualize This: the FlowingData Guide to Design,Visualization, and Statistics. Nathan YauDuke Library’s Introduction to Visualization LibGuideSunlight Foundation’s Visualization Style GuideData Visualization Catalogue

47

NODE.JS RESOURCESAbout Node.js

Open-source, cross-platform JavaScript runtimeenvironment for developing server-side WebapplicationsUses , implemented in C++Runs on: Windows XP and later, Mac OS X 10.5+, LinuxServer-side, Event-driven, Asynchronous, Many modulesavailable

Node.js @ Lynda.com

Jim Wilson Cantelon, Harter, Holowaychuk, Rajlich

Chrome V8 JavaScript engine

Node.js home page and docsFelix's Node Guide

Node Beginner BookNode.js the Right Way.Node.js in Action.