MK99 – Big Data 1

Big data &

cross-platform analytics MOOC lectures Pr. Clement Levallois

MK99 – Big Data 2

Data visualization

Data Visualization

MK99 – Big Data 3

“The ability to collect, store, and manage data is increasing quickly, but our ability to understand it remains constant.”

Ben Fry Co-creator or Processing and Principal at Fathom

The problem to be solved

MK99 – Big Data 4

“The picture-examining eye is the best finder we have of the wholly unanticipated.” “ [Exploratory Data Analysis] is an approach to analyzing datasets to summarize their main characteristics in easy-to-understand form, often with visual graphs, without using a statistical model or having formulated a hypothesis”

Data visualization: inspired by John Tukey’s “Exploratory Data Analysis”

Tukey et al., 1983

MK99 – Big Data 5

Dataviz draws from different communities

Information visualization /

Human Computer Interaction

how do humans interact with the technology?

Cartography focus on

Geographical Information

Systems

Information design

What are the best designing practice?

MK99 – Big Data 6

The mother of all modern dataviz? • “The map of the market”

• Created in 1998 for smartmoney.com

• Still online and used!

• Author: Matt Wattenberg

• How to read it

– Colors represent change in stock price – Surface represents market capitalization

• What it does – Evolution by sectors at a glance – Facilitates comparisons / outlier detection – Drilling: clicking on a sector zooms on it.

MK99 – Big Data 7

Key examples of dataviz • 2010: Memorial for the victims of the Twin Towers

– http://blog.blprnt.com/blog/blprnt/all-the-names/picture-2-3 – Author: Jer Thorp (www.blprnt.com)

• 2011: OECD Better Life Index – http://www.oecdbetterlifeindex.org/ – Author: Moritz Stefaner (www.truth-and-beauty.net)

• 2012: Realtime map of wind speed in the US

– http://hint.fm/wind/ – Authors: Fernanda Viégas and Martin Wattenberg (www.hint.fm)

• 2013: The story of every known drone strike and victim in Pakistan.

– http://drones.pitchinteractive.com/ – Authors: Pitch Interactive (http://pitchinteractive.com/)

MK99 – Big Data 8

Twin Tower Memorial

MK99 – Big Data 9

OECD Better Life Index

MK99 – Big Data 10

MK99 – Big Data 11

Drone strikes and victims, 2005-2013

MK99 – Big Data 12

What makes a good data visualization?

1. A pre-condition: data must be respected – no photoshopping!

2. A « trademark »: data remains largely disaggregated

3. A result: the audience learns something new, via a process of discovery

4. A follow-up: the experience is « addictive », it is memorable and one gets drawn back to it.

MK99 – Big Data 13

Ben Fry’s process of creation in 7 steps

From Fry (2005) PhD dissertation: Computational Information Design available at: benfry.com/phd/

MK99 – Big Data 14

Data visualization: what use for business?

• Exploration – Is there any interesting pattern / segment to be found in my customer data?

• Communication

– Here is the visual summary of my conclusions, for my manager – Here is a visual explanation of the analytical solution I push to the client

• Control

– Monitoring monthly indicators and trends for production, sales etc. on a dashboard

• Emotion

– Conveying a critical message in a communication campaign through a powerful visualization

MK99 – Big Data 15

Custom

Solutions to create data visualizations

• By hand

• Using a pro / small agency

• Bigger agencies

• Packaged tools

One-off projects Data of limited size Bleeding edge design

Large set of features Fit for big data Standard design Can deploy to the entire org

Small agencies Teams inside orgs

BI solutions

Off the shelf

MK99 – Big Data 16

1. Data processing – R, Python and Matlab are favored both in the academic community & in the industry – Java is used heavily for data management in companies, also used in academia for data-

intensive tasks.

2. Visualization – Processing (Java-based): to create videos or installations. – Javascript (D3.js, three.js, Google Chart API, SigmaJS etc.): to create web-based animations – Flash (Flash itself or Flare): to create desktop applications. – R and Python (some packages): to create pictures – often “scientific” in style.

3. Gimp, Inkscape, PhotoShop or Illustrator to polish the result for print outputs.

1. Using programmatic tools HOW

MK99 – Big Data 17

But why would I bother with programming? -> You get much more flexibility and creativity.

-> Here is just a sample of what can be done in javascript with D3js.org

MK99 – Big Data 18

2. Hiring an agency

A choice of 8 outstanding solutions

www.tulpinteractive.com (NL) www.truth-and-beauty.net (DE) www.periscopic.com (US, Portland) www.interactivethings.com/ (CH)

www.o-c-r.org (US, NYC) www.dataveyes.com (FR) www.fathom.info (US, Boston) www.pitchinteractive.com (US, Oakland)

MK99 – Big Data 19

• Gephi

• NodeXL

• VosViewer

3. “click and point” applications

• ArcGIS

• QGIS

• MapBox

• Google Fusion Tables

• Tableau

• Synerscope

• joliCharts

• Plot.ly

• Excel

• Data Wrapper

• Raw http://app.raw.densitydesign.org/

Drawing networks Drawing maps Drawing networks, charts and / or maps

MK99 – Big Data 20

4. BI integrated solutions

• Tableau

• Qlik Sense

• BIME Analytics

• Palantir

• Spotfire

• SiSense

MK99 – Big Data 21

5. To go further… • Critical thinking on data visualization

– http://datastori.es -> A podcast series

– VizWiz

– the Why Axis

– Junk Charts

– WTF Visualizations

• Great lists of tools and resources

– Visualisingdata.com

– Datavisualization.ch

MK99 – Big Data 22

This slide presentation is part of a course offered by EMLYON Business School (www.em-lyon.com).

Contact Clement Levallois (levallois [at] em-lyon.com) for more information.