strata ny sep 2011: big data, short urls: learning in realtime

41
Short URLs, Big Data: Learning about the world in realtime Hilary Mason [email protected] @hmason

Upload: hilary-mason

Post on 04-Jul-2015

21.852 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Short URLs, Big Data:

Learning about the world in realtime

Hilary Mason

[email protected] @hmason

Page 2: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 3: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

[nyc]

Page 4: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

[NY: City of Data]

http://smrt.io/rmcZOU

Page 5: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 6: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 7: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Dirty Secret of Social Media:

It’s all GOSSIP

Page 8: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Timely

Crosses networks

Tells us about the world

Page 9: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 10: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 11: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

[dystopia]

Page 12: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 13: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

realtime?!

Page 14: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 15: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

realtime, baby!

Page 16: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 17: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Realfast?

Page 18: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 19: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Infrastructure

Page 20: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

metadata > data

Page 21: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

What languages are in a page?

Page 22: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

raw data

"es"

"en-us,en;q=0.5"

"pt-BR,pt;q=0.8,en-US;q=0.6,en;q=0.4"

"en-gb,en;q=0.5"

"en-US,en;q=0.5"

"es-es,es;q=0.8,en-us;q=0.5,en;q=0.3”

"de, en-gb;q=0.9, en;q=0.8"

Page 23: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 24: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

http://4sq.com/96kc1O

Page 25: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

What are the languages spoken

by the people who are

interested in the page?

Page 26: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

How long do people pay

attention to things?

Page 27: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 28: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 29: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 30: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 31: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 32: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

How does one idea relate to the

next?

Page 33: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 34: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Gossip ... with queries

Page 35: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 36: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 37: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 38: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Timely

Crosses networks

Tells us about the world

Page 39: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

Simple

Fast

Reduces friction

Page 40: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime
Page 41: Strata NY Sep 2011: Big Data, Short URLs: Learning in Realtime

[email protected]

@hmason

Thank you!