data visualization summary ihub

Data Visualization Nikhil Srivastava, 2015

Nikhil Srivastava

iHub Summer Data Jam

About this Lecture

• Shortened version of longer course

• Course website

– Slides, demos, extra material

– Code samples and libraries

– Final projects

Effective Data Visualization

Nikhil Srivastava

nsrivast@gmail.com

0713 987 262

I build products & businesses in the fields of finance & technology.

I organize & visualize information for teaching & understanding.

nikhilsrivastava.com

• What is Data Visualization?

• Thinking and Seeing

• From Data to Graphics

• Principles and Guidelines

• Building Visualizations

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

Outline

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

Data Visualization

Information Visualization

Scientific Visualization Infographics

Statistical GraphicsInformative Art

ArtScience

Statistics

JournalismDesign

Visual Analytics

City/Town County Population Ahero Kisumu 76,828 Athi River Machakos 139,380 Awasi Kisumu 93,369 Kangundo-Tala Machakos 218,557 Karuri Kiambu 129,934 Kiambu Kiambu 88,869 Kikuyu Kiambu 233,231 Kisumu Kisumu 409,928 Kitale Trans-Nzoia 106,187 Kitui Kitui 155,896 Limuru Kiambu 104,282 Machakos Machakos 150,041 Molo Nakuru 107,806 Mwingi Kitui 83,803 Naivasha Nakuru 181,966 Nakuru Nakuru 307,990 Nandi Hills Trans-Nzoia 73,626 Ruiru Kiambu 238,858 Thika Kiambu 139,853

• Which is the most populous

city in the list?

• Which county in the list has

the most cities?

the largest average city?

city in the list?

the most cities?

city in the list?

the most cities?

• What is the population of

Limuru?

city in the list?

the most cities?

Data Visualization is:

• Useful

– Answers user questions

– Reduces user workload

(by design, not by default)

Anscombe’s quartet (1973)

• Important

– Understand structure and patterns

– Resolve ambiguity

– Locate outliers

• Important

– Design decisions affect interpretation

• Powerful

– Communicate, teach, inspire

• Relevant

– In one second …

– Open data, open technologies

– Growing use in business,

education, media, advertising …

Definitions

• “the process that transforms (abstract) data into

interactive graphical representations” 1

• “finding the artificial memory that best supports

our natural means of perception” 2

• “visual representations of data to amplify

cognition” 3

• “giving information a visual representation” 4

Focus Extra

purpose communicate explore, analyze

data numerical,categorical

text, maps, graphs, networks

feature representation animation,Interactivity

Course Scope

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

Bandwidth of Our Senses

Why Vision?

The Hardware

The Software• High-level concepts: objects,

symbols

• Involves working memory

• Slower, serial, conscious

• Sensory input

• Low-level features: orientation,

shape, color, movement

• Rapid, parallel, automatic

Visual Perception

“Bottom-up”

The Software• High-level concepts: objects,

symbols

• Involves working memory

• Slow, sequential, conscious

• Sensory input

• Low-level features: orientation,

shape, color, movement

• Rapid, parallel, automatic

Visual Perception

“Bottom-up”

“Top-down”

Task: Counting

How many 3’s?

1281768756138976546984506985604982826762 9809858458224509856458945098450980943585 9091030209905959595772564675050678904567 8845789809821677654876364908560912949686

Task: Counting

How many 3’s?

1281768756138976546984506985604982826762 9809858458224509856458945098450980943585 9091030209905959595772564675050678904567 8845789809821677654876364908560912949686

Task: Counting

Slow, sequential, conscious

Rapid, parallel, automatic

1281768756138976546984506985604982826762 9809858458224509856458945098450980943585 9091030209905959595772564675050678904567 8845789809821677654876364908560912949686

Task: (Distractor) Search

Which side has the red circle?

Task: (Distractor) Search

Task: Search

Slow, sequential, conscious

Task: Unique SearchSlow, sequential, conscious

Lessons for Visualization

• Use “pre-attentive” attributes when possible

– Color, shape, orientation (depth, motion)

– Faster, higher bandwidth

• Caveats

– Working memory: magical number 7 (+/- 2)

– Be careful mixing attributes

Example: Too Many Attributes

Eye != Camera

limited aperture

limited color

Eye != Camera

Saccades: limited time and location

Eye != Camera: Relative

Eye != Camera: Knowledge

Lessons for Visualization

• Human vision has limits and constraints:

aperture, color, time, location

• “What we see” depends on “what we

know”

• Attention and experience matters

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

From Data to Graphics

What kind

of data do

we have?

How can we

represent the

data visually?

How can we

organize this into

a visualization?

Athi River Machakos 139,380

Awasi Kisumu 93,369

Kangundo-Tala Machakos 218,557

Karuri Kiambu 129,934

Kiambu Kiambu 88,869

Kikuyu Kiambu 233,231

Kisumu Kisumu 409,928

Kitale Trans-Nzoia 106,187

Kitui Kitui 155,896

Limuru Kiambu 104,282

Machakos Machakos 150,041

Molo Nakuru 107,806

Mwingi Kitui 83,803

Naivasha Nakuru 181,966

Nakuru Nakuru 307,990

Nandi Hills Trans-Nzoia 73,626

Visual Encoding

What kind

of data do

we have?

How can we

represent the

data visually?

How can we

organize this into

a visualization?

Awasi Kisumu 93,369

Kitui Kitui 155,896

Molo Nakuru 107,806

Mwingi Kitui 83,803

Data as Input

Awasi Kisumu 93,369

Kitui Kitui 155,896

Molo Nakuru 107,806

Mwingi Kitui 83,803

CleanRestructure

ExploreAnalyze

Visualization Goals

Model and Attribute

item attribute A attribute B … attribute M

item 1 value1_A value1_B …

item 2 value2_A value2_B …

… … …

item N valueN_M

Data TypesCATEGORICAL ORDINAL NUMERICAL

Interval Ratio

Male / Female

Asia / Africa / Europe

True / False

Small / Med / Large

Low / High

Yes / Maybe / No

Latitude/Longitude

Compass direction

Time (event)

Length

Time (duration)

= = = =

< > < > < >

+ - + -

Data Types: Example

• Which are categorical? (=)

• Which are ordinal? (= < >)

ID Gender Test Score Grade Size Temperature

1 Male 77 C Small 36.5

2 Female 85 B Large 37.2

3 Female 95 A Medium 36.7

4 Male 90 A Large 37.4

• Which are interval? (= < > + -)

• Which are ratio? (= < > + - * /)

Data Type TransformationCATEGORICAL ORDINAL NUMERICAL

Interval Ratio

Male / Female

Asia / Africa / Europe

True / False

Small / Med / Large

Low / High

Yes / Maybe / No

Latitude/Longitude

Compass direction

Length

Binning/Categorizing

Differencing/Normalization

Advanced Data Types

• Networks/Graphs

– Hierarchies/Trees

• Text

• Maps: points, regions, routes

What kind

of data do

we have?

How can we

represent the

data visually?

How can we

organize this into

a visualization?

Awasi Kisumu 93,369

Kitui Kitui 155,896

Molo Nakuru 107,806

Mwingi Kitui 83,803

Visual Encodings

volume

Channels

position

angle/tilt

Channel Effectiveness

“Spatial position is such a good visual

coding of data that the first decision of

visualization design is which variables get

spatial encoding at the expense of others”

Color as a Channel

Categorical Quantitative

Hue Good (6-8 max)

Value Poor Good

Saturation Poor Okay

What kind

of data do

we have?

How can we

represent the

data visually?

How can we

organize this into

a visualization?

Awasi Kisumu 93,369

Kitui Kitui 155,896

Molo Nakuru 107,806

Mwingi Kitui 83,803

type mark channel data represented

Scatter Plot point position 2 quantitative

Scatter + Hue point position,color

2 quantitative, 1 categorical

Scatter + Size (“Bubble”)

point position,size

3 quantitative

Scatter Plot – Applications

CORRELATION GROUPING OUTLIERS

Scatter Plot – Dangers

OCCLUSION (DENSITY)

OCCLUSION (OVERLAP)

Line Chart line position(orientation)

2 quantitative

Area Chart area size (length) 2 quantitative

Line Chart – Applications

PATTERN OVER TIME COMPARISON

Line Chart – Dangers

Y SCALING

X SCALING

OVERLOAD

Bar Chart line size (length) 1 categorical,1 quantitative

Histogram line size (length) 1 ordinal/quantitative,1 quantitative (count)

Bar Chart – Applications

COMPARE CATEGORIES DISTRIBUTION

Bar Chart – Dangers

TOO MANY CATEGORIES

POORLY SORTED

Pie Chart area size (angle) 1 quantitative

Pie Chart – Dangers

AREA SCALE SIMILAR AREAS OVERLOAD

Multi-Series Bar Charts

GROUPED BAR CHART

STACKED BAR CHART

Multi-Series Line Charts

MULTIPLE LINE

STACKED AREA CHART

Normalization

NORMALIZED BAR NORMALIZED AREA

Small Multiples Chart

Advanced Charts

Treemap (Hierarchical Data)

Channels: ?

Strengths:

nested relationships

Concerns:

order vs aspect ratio

Advanced Charts

Multi-Level Pie(Hierarchical Data)

Channels: ?

Strengths:

nested relationships

Concerns:

readability

Advanced Charts

Heat Map(Table/Field Data)

Channels: ?

Strengths: pattern/outlier detection

Concerns: ordering/ clustering

Advanced Charts

Choropleth Map(Region Data)

Channels: ?

Strengths:

geography

Concerns:

region size

color spectrum

Advanced Charts

Cartogram(Region Data)

Channels: ?

Strengths: geographic pattern

Concerns: base map knowledge

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

From Science to Art

• Design principles*

• Style guidelines*

*dependent on visualization context

and objective (and author)

Design Principles

• Integrity

– Tell the truth with data

• Effectiveness

– Achieve visualization objectives

• Aesthetics

– Be compelling, vivid, beautiful

Integrity

Lie Ratio = size of effect in graphic

size of effect in data

Integrity

“show data variation, not design variation”

Effectiveness*

Data/Ink Ratio = ink representing data

total ink

*according to Tufte

Effectiveness* *according to Tufte

avoid “chart junk”

Avoid Chart Junk

Effectiveness (Few)

Practical Guidelines

• Avoid 3-D charts

• Focus on substance over graphics

• Avoid separate legends and keys

• Faint grids/guidelines

• Avoid unnecessary textures and colors

Color Guidelines

• To label

• To emphasize

• To liven or decorate

Bad Color

Good Color

More Color Guidelines

• Use color only when necessary

• Use saturated colors for data labels, thin

lines, small areas

• Use less saturated colors for large areas,

backgrounds

• Use tools like ColorBrewer

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

What Software to Use?

Awasi Kisumu 93,369

Kitui Kitui 155,896

Molo Nakuru 107,806

Mwingi Kitui 83,803

CleanRestructure

ExploreAnalyze

Visualization Goals

Visualization Software

• Web friendly

– Highcharts

– InfoVis

– Processing

– D3

• Statistics

– Python (Matplotlib)

– R (ggplot2)

• Maps

– Google Charts

– Leaflet

– CartoDB

• Dashboards

• Graphs

– GraphViz

– Gephi

Highcharts - Reference

• Examples

– Hello, Chart

– Basic Charts

• Documentation, API

• Highcharts Cloud

• Advanced Topics

introduction

foundation & theory

building blocks

design & critique

construction

advanced

The Ebb and Flow of Movies

NY Times, 2008

Advanced Visualizations

Word Cloud - “Data Visualization” Wikipedia PageWordle

ZIPScribbleRobert Kosara, 2006

Twitter NetworksPJ Lamberson, 2012

• Infosthetics.com

• Visualizing.org

• FlowingData.com

Nikhil Srivastava

nsrivast@gmail.com

data visualization summary ihub

effective data visualization

kiambu kiambu

limuru kiambu

nakuru nakuru

machakos machakos

kisumu kisumu

kikuyu kiambu

ruiru kiambu

Education

ihub - o ovr · 2015-01-30 · ihub research is the...

data visualization in review: summary report

data science and visualization - ihub · data science and...

california innovation hub (ihub) - usccalifornia innovation...

about integrating applications into birt ihub

f a c t s @ ihub

using visualization platform -...

intellectual property in technological innovations - ihub

ihub // about

data visualization in review: summary report - idrc...

deploying to a birt ihub system

auc ihub · 2019-11-12 · auc ihub connecting creative...

opentext information hub (ihub) 3.1 and 3.1 ·...

ihub model report final

inland so cal ihub 02.2014

ihub python course brochure

ted - data visualization - summary and conversation lesson

savannah fund overview @ ihub

system administration guide -...

ihub cloud service (ihub cs - inspirage · 2017-02-07 ·...