adventure in data: a tour of visualization projects at twitter
TRANSCRIPT
![Page 1: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/1.jpg)
Krist Wongsuphasawat / @kristw
Adventure in dataA tour of visualizations at Twitter
![Page 2: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/2.jpg)
Krist Wongsuphasawat / @kristw
![Page 3: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/3.jpg)
Computer EngineerBangkok, Thailand
PhD in Computer ScienceUniv. of MarylandInformation Visualization
IBMMicrosoft
Data Visualization ScientistTwitter
Krist Wongsuphasawat / @kristw
![Page 4: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/4.jpg)
Krist Wongsuphasawat / @kristw
Adventure in dataA whirlwind tour of visualization projects at Twitter
![Page 5: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/5.jpg)
Get data
![Page 6: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/6.jpg)
![Page 7: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/7.jpg)
Having all Tweets
How people think I feel.
![Page 8: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/8.jpg)
How people think I feel. How I really feel.
Having all Tweets
![Page 9: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/9.jpg)
• Too much data. Want only relevant Tweets
• hashtag: #BRA
• keywords: “goal”
• Need to aggregate & reduce size
• Long processing time (hours)
Challenges
![Page 10: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/10.jpg)
Hadoop ClusterData Storage
Workflow
![Page 11: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/11.jpg)
Hadoop Cluster
Pig / Scalding (slow)
Data Storage
Tool
Workflow
![Page 12: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/12.jpg)
Hadoop Cluster
Pig / Scalding (slow)
Data Storage
Tool
Workflow
![Page 13: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/13.jpg)
Hadoop Cluster
Pig / Scalding (slow)
Data Storage
Tool
Smaller datasetYour laptop
Workflow
![Page 14: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/14.jpg)
Hadoop Cluster
Pig / Scalding (slow)
Data Storage
Tool
Final dataset
Tool node.js / python / excel (fast)
Your laptop
Workflow
Smaller dataset
![Page 15: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/15.jpg)
Krist Wongsuphasawat / @kristw
Cleaning dataa story of my life
![Page 16: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/16.jpg)
Storytelling
Analytics Tools
Creative
Projects
![Page 17: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/17.jpg)
To understand the world and share the stories
To understand Twitter users and improve the service
To showcase the data and inspire
Projects
Storytelling
Analytics Tools
Creative
![Page 18: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/18.jpg)
Storytelling1
World Cup ElectionOscars
TV Shows New YearEarthquake
Super BowlProtest
…
BehaviorsSleepingDaylight savingLanguage
…
Events
FastingInformation spread
Commute
![Page 19: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/19.jpg)
So many things we could learn
from Twitter data
![Page 20: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/20.jpg)
Give us interesting vis about xxxx by Nov 10
![Page 21: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/21.jpg)
Challenge accepted
![Page 22: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/22.jpg)
Tweets
(+ media)photos, videos
What?
Where? When?
GEO TIME
TEXT
![Page 23: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/23.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 24: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/24.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 25: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/25.jpg)
Time Tweets/second
![Page 26: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/26.jpg)
Time Tweets/second
![Page 27: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/27.jpg)
Time Tweets/second + Annotation
http://www.flickr.com/photos/twitteroffice/5681263084/
![Page 28: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/28.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 29: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/29.jpg)
GeoHeatmap
Low density
High density
![Page 30: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/30.jpg)
GeoSan Francisco
flickr.com/photos/twitteroffice/8798020541
Low density
High density
![Page 31: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/31.jpg)
GeoSan Francisco
Rebuild the world based on
tweet volumes
twitter.github.io/interactive/andes/
![Page 32: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/32.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 35: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/35.jpg)
Text Word cloud of Tweets right after the 1st goal
www.wordle.net
It was an “own” goal.
![Page 36: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/36.jpg)
Text WordTree [Wattenberg & Viégas 2008]www.jasondavies.com/wordtree
www.jasondavies.com/wordtree
![Page 37: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/37.jpg)
Text
word/phrase/hashtag count topic
![Page 38: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/38.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 39: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/39.jpg)
Time + Geo Tweet pattern [Rios & Lin 2012]
Night
Late night
Daytime
Night
Late night
Daytime
![Page 40: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/40.jpg)
Night
Late night
Daytime
Night
Late night
Daytime
Time + Geo Tweet pattern [Rios & Lin 2012]
![Page 41: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/41.jpg)
Night
Late night
Daytime
Night
Late night
Daytime
Time + Geo Tweet pattern [Rios & Lin 2012]
![Page 42: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/42.jpg)
Night
Late night
Daytime
Night
Late night
Daytime
Time + Geo Tweet pattern [Rios & Lin 2012]
![Page 43: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/43.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 44: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/44.jpg)
Geo + Text Real-time Tweet map
![Page 45: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/45.jpg)
Geo + Text Real-time Tweet map
![Page 46: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/46.jpg)
most frequent
term
Geo + Text Real-time Tweet map
![Page 47: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/47.jpg)
Gmail was down Jan 24, 2014
Geo + Text Real-time Tweet map
![Page 48: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/48.jpg)
Nelson Mandela passed away Dec 5, 2013
Geo + Text Real-time Tweet map
![Page 49: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/49.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 50: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/50.jpg)
Time + Text
UEFA Champions League
Biggest tournament for European soccer clubs
Many Tweets during the matches
![Page 51: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/51.jpg)
UEFA Champions League
Dortmund Bayern Munich
Count Tweets mentioning the teams every minute
Team 1 Team 2
Time + Text
![Page 52: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/52.jpg)
Time + Text UEFA Champions League
![Page 53: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/53.jpg)
+ “goal” count + context
Time + Text UEFA Champions League
![Page 54: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/54.jpg)
+ “offside”
Time + Text UEFA Champions League
![Page 55: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/55.jpg)
+ players
Time + Text UEFA Champions League
![Page 57: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/57.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 58: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/58.jpg)
Time + Text + Geo State of the Union
twitter.github.io/interactive/sotu2014
![Page 59: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/59.jpg)
1) timeline + topic from Tweets
4) Density map of Tweets about selected topic
3) Volume of Tweets by topics
during selected part of the SOTU
2) context (speech)
twitter.github.io/interactive/sotu2014
Time + Text + Geo State of the Union
![Page 60: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/60.jpg)
World Cup 2014Time + Textinteractive.twitter.com/wccompetitree
![Page 61: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/61.jpg)
Time + Text + Geo World Cup 2014
interactive.twitter.com/wccompetitree
![Page 62: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/62.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
![Page 63: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/63.jpg)
What?
Where? When?
GEO TIME
TEXT
Visualize Tweets
+Non-Twitter data
CONTEXT
![Page 64: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/64.jpg)
Time + Text New Year 2014
![Page 65: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/65.jpg)
Time + Text New Year 2014
![Page 66: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/66.jpg)
Time + Text + Geo (c) New Year 2014
twitter.github.io/interactive/newyear2014/
![Page 67: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/67.jpg)
Analytics Tools2Data sources
Output
explore
analyze
present
get
*
*
![Page 68: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/68.jpg)
Analytics Tools2Data sources
Output
explore
analyze
present
get
*
*
![Page 69: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/69.jpg)
Analytics Tools2Data sources
Output
explore
analyze
present
get
*
*
ad-hoc scripts
![Page 70: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/70.jpg)
Analytics Tools2Data sources
Output
explore
analyze
present
get
*
*
ad-hoc scripts tools for exploration
![Page 71: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/71.jpg)
User activity logs
![Page 72: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/72.jpg)
UsersUseTwitter
![Page 73: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/73.jpg)
UsersUse
Product Managers
Curious
![Page 74: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/74.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop Write Twitter
Instrument
Product Managers
![Page 75: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/75.jpg)
What are being logged?
tweetactivities
![Page 76: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/76.jpg)
What are being logged?
tweet from home timeline on twitter.com tweet from search page on iPhone
activities
![Page 77: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/77.jpg)
What are being logged?
tweet from home timeline on twitter.com tweet from search page on iPhone
sign up log in
retweet etc.
activities
![Page 78: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/78.jpg)
Organize?
![Page 79: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/79.jpg)
log event a.k.a. “client event”
[Lee et al. 2012]
![Page 80: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/80.jpg)
log event a.k.a. “client event”
client : page : section : component : element : actionweb : home : timeline : tweet_box : button : tweet
1) User ID 2) Timestamp 3) Event name
4) Event detail
[Lee et al. 2012]
![Page 81: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/81.jpg)
Log data
![Page 82: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/82.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop Twitter
Instrument
Write
Product Managers
bigger than Tweet data
![Page 83: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/83.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop
Data Scientists
Ask
Instrument
Write
Product Managers
![Page 84: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/84.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop
Data Scientists
Find
Ask
Instrument
Write
Product Managers
![Page 85: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/85.jpg)
Log data
![Page 86: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/86.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop
Data Scientists
Find, Clean
Ask
Instrument
Write
Product Managers
![Page 87: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/87.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop
Data Scientists
Find, Clean
Ask
Monitor
Instrument
Write
Product Managers
![Page 88: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/88.jpg)
UsersUse
Curious
Engineers
Log datain Hadoop
Data Scientists
Find, Clean, Analyze
Ask
Monitor
Instrument
Write
Product Managers
![Page 89: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/89.jpg)
Log data
EngineersData Scientists
Usersin Hadoop
Find, Clean, Analyze
Use
Monitor
Ask
Curious
1 2
Instrument
Write
Product Managers
![Page 90: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/90.jpg)
Part I Find & Monitor Client Events
![Page 91: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/91.jpg)
Motivation
![Page 92: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/92.jpg)
Log datain Hadoop
Engineers & Data Scientists
billions of rows
![Page 93: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/93.jpg)
Log datain Hadoop
Aggregate
10,000+ event types
date client page section comp. elem. action count
20141011 web home home - - impression 100
20141011 web home wtf - - click 20
Engineers & Data Scientists
Client event collection
![Page 94: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/94.jpg)
Log datain Hadoop
Aggregate
10,000+ event types
date client page section comp. elem. action count
20141011 web home home - - impression 100
20141011 web home wtf - - click 20
Engineers & Data Scientists
Client event collection
(Who-to-Follow)
![Page 95: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/95.jpg)
Log datain Hadoop
AggregateClient event collection
Engineers & Data Scientists
![Page 96: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/96.jpg)
Log datain Hadoop
Aggregate
Find
client page section component element action
Search
Client event collection
Engineers & Data Scientists
![Page 97: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/97.jpg)
Log datain Hadoop
Aggregate
Find
client page section component element action
Search
Client event collection
Engineers & Data Scientists
![Page 98: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/98.jpg)
section? component?
element?
![Page 99: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/99.jpg)
client page section component element action
Search
Find
Log datain Hadoop
Aggregate
web home * * impression*
Client event collection
Engineers & Data Scientists
![Page 100: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/100.jpg)
client page section component element action
Search
Find
Query
Return
Log datain Hadoop
Resultsweb : home : home : - : - : impression
web : home : wtf : - : - : impression
Aggregate
web home * * impression*
Client event collection
Engineers & Data Scientists
![Page 101: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/101.jpg)
client page section component element action
Search
Find
Query
Return
Log datain Hadoop
Resultsweb : home : home : - : - : impression
web : home : wtf : - : - : impression
Aggregate
search can be better
Client event collection
Engineers & Data Scientists
![Page 102: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/102.jpg)
client page section component element action
Search
Find
Query
Return
Log datain Hadoop
Resultsweb : home : home : - : - : impression
web : home : wtf : - : - : impression
Aggregate
10,000+ event types
search can be better
Client event collection
Engineers & Data Scientists
![Page 103: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/103.jpg)
client page section component element action
Search
Find
Query
Return
Log datain Hadoop
Resultsweb : home : home : - : - : impression
web : home : wtf : - : - : impression
Aggregate
search can be better
10,000+ event types
not everybody knowsWhat are all sections under web:home?
Client event collection
Engineers & Data Scientists
![Page 104: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/104.jpg)
client page section component element action
Search
Find
Query
Return
Log datain Hadoop
Resultsweb : home : home : - : - : impression
Aggregate
search can be better
one graph / event
10,000+ event types
not everybody knowsWhat are all sections under web:home?
Client event collection
Engineers & Data Scientists
![Page 105: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/105.jpg)
client page section component element action
Search
Find
Query
Return
Log datain Hadoop
Resultsweb : home : home : - : - : impression
Aggregate
search can be better
one graph / eventx 10,000
10,000+ event types
not everybody knowsWhat are all sections under web:home?
Client event collection
Engineers & Data Scientists
![Page 106: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/106.jpg)
!
• Search for client events
• Explore client event collection
• Monitor changes
Goals
![Page 107: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/107.jpg)
Design
![Page 108: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/108.jpg)
Client event collection
Engineers & Data Scientists
![Page 109: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/109.jpg)
See
Client event collection
Engineers & Data Scientists
![Page 110: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/110.jpg)
See
Interactions search box => filter
Client event collection
narrow down
Engineers & Data Scientists
![Page 111: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/111.jpg)
See
How to visualize?
narrow down
Client event collection
Engineers & Data Scientists
Interactions search box => filter
![Page 112: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/112.jpg)
See
How to visualize?
narrow down
Client event collection
Engineers & Data Scientists
client : page : section : component : element : actionInteractions search box => filter
![Page 113: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/113.jpg)
Client event hierarchy
iphone home -
- - impression
tweet tweet click
iphone:home:-:-:-:impressioniphone:home:-:tweet:tweet:click
![Page 114: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/114.jpg)
Detect changes
iphone home -
- - impression
tweet tweet click
iphone home -
- - impression
tweet tweet click
TODAY
7 DAYS AGO
compared to
![Page 115: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/115.jpg)
Calculate changes
+5% +5% +5%
+10% +10% +10%
-5% -5% -5%
DIFF
![Page 116: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/116.jpg)
Display changes
iphone home -
- - impression
tweet tweet click
Map of the Market [Wattenberg 1999], StemView [Guerra-Gomez et al. 2013]
![Page 117: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/117.jpg)
Display changes
home -
- - impression
tweet tweet click
iphone
![Page 118: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/118.jpg)
Demo Scribe Radar
![Page 119: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/119.jpg)
Twitter for Banana
![Page 120: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/120.jpg)
![Page 121: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/121.jpg)
Part II Analysis
![Page 122: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/122.jpg)
Count page visits
banana : home : - : - : - : impressionhome page
![Page 123: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/123.jpg)
Funnel
home page
profile page
![Page 124: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/124.jpg)
Funnel analysis
banana : home : - : - : - : impression
banana : profile : - : - : - : impression
1 jobhome page
profile page
1 hour
![Page 125: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/125.jpg)
Funnel analysis
banana : home : - : - : - : impression
banana : profile : - : - : - : impression banana : search : - : - : - : impression
home page
profile page search page
2 jobs2 hours
![Page 126: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/126.jpg)
Funnel analysis
banana : home : - : - : - : impression
banana : profile : - : - : - : impression banana : search : - : - : - : impression
home page
profile page search page
Specify all funnels manually!
n jobsn hours
![Page 127: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/127.jpg)
Goal
banana : home : - : - : - : impression
… ……
1 job => all funnels, visualized
home page
![Page 128: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/128.jpg)
• Visualize an overview of event sequences
!
• Big data? eBay checkout sequences
Related work
[Wongsuphasawat et al. 2011, Monroe et al. 2013, …]
[Shen et al. 2013]
![Page 129: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/129.jpg)
User sessionsSession#1
A
B
start
end
Session#4
start
end
A
Session#2
B
start
end
A
Session#3
C
start
end
A
![Page 130: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/130.jpg)
Aggregate4 sessions
A
BB C
start
end endend
A A
end
A
![Page 131: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/131.jpg)
Aggregate
A
BB C
start
end endend
end
4 sessions
![Page 132: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/132.jpg)
Aggregate
C
start
end endend
end
A
B
4 sessions
![Page 133: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/133.jpg)
Aggregate
C
start
end endend
end
A
B
4 sessions
![Page 134: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/134.jpg)
Aggregate
C
start
end endend
A
B end
4 sessions
![Page 135: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/135.jpg)
Aggregate
C
start
endend
A
B end
4 sessions
![Page 136: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/136.jpg)
Aggregate
C
start
endend
A
B end
4 sessions
![Page 137: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/137.jpg)
Aggregate
start
endend
A
CB end
4 sessions
![Page 138: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/138.jpg)
Aggregate
4,000,000 sessions
endend
A
CB end
start
![Page 139: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/139.jpg)
Twitter for Banana
![Page 140: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/140.jpg)
try with sample data (~millions sessions, 10,000+ event types)
!
original paper (100,000 sessions, ~10 event types)
![Page 141: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/141.jpg)
fail…
![Page 142: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/142.jpg)
How to make it work?
![Page 143: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/143.jpg)
# of unique sequences
![Page 144: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/144.jpg)
1. Reduce event types
Reduce # of unique sequences
![Page 145: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/145.jpg)
1. Reduce event types
Reduce # of unique sequences
10,000 types select
tweet sign up log out
![Page 146: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/146.jpg)
1. Reduce event types
Reduce # of unique sequences
10,000 types select
tweet sign up log out
![Page 147: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/147.jpg)
1. Reduce event types
Reduce # of unique sequences
10,000 types select merge
tweet from home timeline tweet from search page tweet …
= tweet
![Page 148: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/148.jpg)
1. Reduce event types
2. Reduce sequence length
Reduce # of unique sequences
![Page 149: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/149.jpg)
1. Reduce event types
2. Reduce sequence length
Reduce # of unique sequences
session
1000 events
![Page 150: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/150.jpg)
1. Reduce event types
2. Reduce sequence length
Reduce # of unique sequences
session
10 events after (window size & direction)
1000 events
visit home page (alignment)
![Page 151: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/151.jpg)
1. Reduce event types
2. Reduce sequence length
Reduce # of unique sequences
Ask users for input}
![Page 152: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/152.jpg)
1. Reduce event types
2. Reduce sequence length
3. More aggregation on Hadoop
Reduce # of unique sequences
Ask users for input}
![Page 153: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/153.jpg)
Collapse eventsSequence ABBBCCCC ABBCC ABC ABCCCC ABCD ABCCCD ABCCE ABCDF ABCDG ABCDH
e.g. tweet, tweet, tweet, … = tweet
![Page 154: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/154.jpg)
Sequence ABC ABC ABC ABC ABCD ABCD ABCE ABCDF ABCDG ABCDH
Collapse events
![Page 155: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/155.jpg)
Group & CountSequence ABC ABCD ABCE ABCDF ABCDG ABCDH …
Count 2000 80 20 1 1 1 …
![Page 156: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/156.jpg)
Group & CountSequence ABC ABCD ABCE ABCDF ABCDG ABCDH ABCDI ABCDJK ABCDJL
Count 2000 80 20 1 1 1 1 1 1
rare sequences (count < threshold)
![Page 157: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/157.jpg)
TruncateSequence ABC ABCD ABCE ABCDx ABCDx ABCDx ABCDx ABCDJx ABCDJx
Count 2000 80 20 1 1 1 1 1 1
Replace last event with x (…)
![Page 158: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/158.jpg)
Sequence ABC ABCD ABCE ABCDx ABCDJx
Count 2000 80 20 4 2
Group & Count
![Page 159: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/159.jpg)
Truncate moreSequence ABC ABCD ABCE ABCDx ABCDx
Count 2000 80 20 4 2
![Page 160: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/160.jpg)
Group & CountSequence ABC ABCD ABCE ABCDx
Count 2000 80 20 6
![Page 161: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/161.jpg)
1. Define set of events
2. Pick alignment, direction and window size
3. Run Hadoop job (with more aggregation)
4. Wait for it… (2+ hrs)
5. Visualize
Final process
~100,000 patterns (10MB)
gazillion patterns (TBs)
![Page 162: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/162.jpg)
Demo Flying Sessions
![Page 163: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/163.jpg)
• Large-scale User Activity Logs + Visual Analytics
• Used in day-to-day operations at Twitter
• Generalize to smaller systems
Summary
Challenge
big data
small data
visualize & interact
aggregate & sacrifice
![Page 164: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/164.jpg)
Data sources
Output
Creative3
…
![Page 165: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/165.jpg)
https://medium.com/@kristw/designing-the-game-of-tweets-7f87c30dc5a2
Demo / Game of Tweets
![Page 166: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/166.jpg)
To understand the world and share the stories
To understand Twitter users and improve the service
To showcase the data and inspire
Projects
Storytelling
Analytics Tools
Creative
![Page 167: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/167.jpg)
Oh no…. NOT AGAIN
![Page 168: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/168.jpg)
To understand the world and share the stories
To understand Twitter users and improve the service
To showcase the data and inspire
Projects
Storytelling
Analytics Tools
Creative
Reusable Toolkits
To implement once and for all
![Page 169: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/169.jpg)
Coming soon
Demo / Labella.js
![Page 170: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/170.jpg)
https://github.com/twitter/d3kit
Demo / d3Kithttp://www.slideshare.net/kristw/d3kit
![Page 171: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/171.jpg)
ConclusionsData are everywhere.
Many applications: Journalism, Product development, Art, etc.
Combine visualization with other skills: HCI, Design, Stats, ML, etc.
Don’t repeat yourself.
Krist Wongsuphasawat / @kristwinteractive.twitter.com kristw.yellowpigz.com
![Page 172: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/172.jpg)
@philogb @trebor @miguelrios @smrogers @lintool @linuslee @chuangl4
and many other colleagues at @twitter
Acknowledgement
![Page 173: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/173.jpg)
ConclusionsData are everywhere.
Many applications: Journalism, Product development, Art, etc.
Combine visualization with other skills: HCI, Design, Stats, ML, etc.
Don’t repeat yourself.
Krist Wongsuphasawat / @kristwinteractive.twitter.com kristw.yellowpigz.com
![Page 174: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/174.jpg)
Thank you
![Page 175: Adventure in Data: A tour of visualization projects at Twitter](https://reader036.vdocument.in/reader036/viewer/2022070520/58f2533e1a28abe35c8b45dd/html5/thumbnails/175.jpg)
Questions?