eresearch presentation - june '11
TRANSCRIPT
-
8/6/2019 eResearch Presentation - June '11
1/57
Harvesting the Crowd:
Experiments in Crowd-sourcing
Steven Gray
Research AssociateCentre for Advanced Spatial Analysis, UCL
Oxford DTC e-Research Seminar, June 2011
-
8/6/2019 eResearch Presentation - June '11
2/57
Mapper
Ability to ask any question
UK surveys down to postcode or 100metre grid
Realtime mapping
Data visualisation
Export to for integration with existingdatasets
-
8/6/2019 eResearch Presentation - June '11
3/57
Mapper
Born out of MapTube:A map portal developed at the Centre for Advanced Spatial Analysis
-
8/6/2019 eResearch Presentation - June '11
4/57
Mapper
Bespoke Mood Maps for BBC Radio 4, BBC Look East, BBC Look Northand BBC South
-
8/6/2019 eResearch Presentation - June '11
5/57
Mapper
Bespoke Mood Maps for BBC Radio 4, BBC Look East, BBC Look Northand BBC South
-
8/6/2019 eResearch Presentation - June '11
6/57
Mapper
Taking the Concept to the Cloud
Media Organisations
Government
Local Authorities
Community Organisations
Anyone...
-
8/6/2019 eResearch Presentation - June '11
7/57
Mapper
Creating an easy to use interface
Integrating the science behind the scenes
Allowing Global, National, US Zip Codes, Borough, Ward,Postcode or Pin Point maps
Quick and easy data input
Allowing people a free and open view on the the current issues
-
8/6/2019 eResearch Presentation - June '11
8/57
Mapper
-
8/6/2019 eResearch Presentation - June '11
9/57
Mapper.com
World/Nation/City/Borough/
Ward/Street
Survey Anything
Realtime Mapping/DataDownload
Used by The Mayor of London
BBC (currently running - 5000responses in 1 hour)
Scottish Water
The Public....
-
8/6/2019 eResearch Presentation - June '11
10/57
BBC Look East Survey - Broadband Speed Test
-
8/6/2019 eResearch Presentation - June '11
11/57
#5Acts4Wildlife Survey
- First SurveyMapper Survey integrated with our Twitter Collection Tools
-
8/6/2019 eResearch Presentation - June '11
12/57
Twitter API
http://dev.twitter.com/doc
User Profiles Tweet Database
API
http://dev.twitter.com/dochttp://dev.twitter.com/doc -
8/6/2019 eResearch Presentation - June '11
13/57
Search API
HTT
PGET
Req
uest
Resu
lts
{"results":[
{"text":"@twitterapi is great!",
"to_user_id":396524,
"to_user":"TwitterAPI",
"from_user":"jkoum",
"metadata":
{
"result_type":"popular",
"recent_retweets": 100
},
"id":1478555574,
"from_user_id":1833773,
"iso_language_code":"nl",
....}
}
Returns tweets that: match specified query located in Lat/Lon box Historical Tweets User Profile Information Return Public Timeline
Service is rate limited
http://search.twitter.com/search.json?q=@twitterapi JSON or Atom
http://search.twitter.com/search.json?q=@twitterapihttp://search.twitter.com/search.json?q=@twitterapihttp://search.twitter.com/search.json?q=@twitterapihttp://search.twitter.com/search.json?q=@twitterapi -
8/6/2019 eResearch Presentation - June '11
14/57
Streaming API
HTTP Response Stream 3 Levels of Service Different Tracking Services
Track Keywords Geo-Located Tweets Statistically Significant
Public Stream
Limitations
1 connection per IP Address Spritzer is default Stream
400 keywords
25 Lat/Lon Bounding Boxes
-
8/6/2019 eResearch Presentation - June '11
15/57
Streaming API
HTTP Response Stream 3 Levels of Service Different Tracking Services
Track Keywords Geo-Located Tweets Statistically Significant
Public Stream
Limitations
1 connection per IP Address Spritzer is default Stream
400 keywords
25 Lat/Lon Bounding Boxes
-
8/6/2019 eResearch Presentation - June '11
16/57
Streaming API
HTTP Response Stream 3 Levels of Service Different Tracking Services
Track Keywords Geo-Located Tweets Statistically Significant
Public Stream
Limitations
1 connection per IP Address Spritzer is default Stream
400 keywords
25 Lat/Lon Bounding Boxes
-
8/6/2019 eResearch Presentation - June '11
17/57
Twitter Data Collector
-
8/6/2019 eResearch Presentation - June '11
18/57
Tweet-o-Meter
-
8/6/2019 eResearch Presentation - June '11
19/57
Behind the Meters
Gauges Google Gauge [Javascript] 4 servers (Virtualised [Hyper-V]) Each running 4 geo-location searches
30km centriods
Graphs Google LineChart [Javascript] Show 1 hours activity Reset on the hour
-
8/6/2019 eResearch Presentation - June '11
20/57
Behind the Meters
Gauges Google Gauge [Javascript] 4 servers (Virtualised [Hyper-V]) Each running 4 geo-location searches
30km centriods
Graphs Google LineChart [Javascript] Show 1 hours activity Reset on the hour
-
8/6/2019 eResearch Presentation - June '11
21/57
Public Status Sample Stream
2 days 13 hours - 1,319,534 tweetsMonday 25th 2010 18:00 - Wednesday 27th 2010 12:30
Mon Tues Wed
-
8/6/2019 eResearch Presentation - June '11
22/57
Worldwide Geo-located Tweets in one weekend
2 days 14 hours - 2,531,287 tweetsFriday 22nd 2010 18:00 - Monday 25th 2010 08:30
Fri Sat Sun Mon
-
8/6/2019 eResearch Presentation - June '11
23/57
San Francisco Geo-located tweets
-
8/6/2019 eResearch Presentation - June '11
24/57
Twitter Messages in London over one Week
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
25/57
Tweets in an Urban Context
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
26/57
Analysis of London Weekend Tweets
-
8/6/2019 eResearch Presentation - June '11
27/57
Data collected
0
175000
350000
525000
700000
NewYork London Paris Munich SanFrancisco Moscow Barcelona SydneyPRO Hong Kong Beijing Cnong Quing Guongzhou
Geo located all
-
8/6/2019 eResearch Presentation - June '11
28/57
Making the Data look nice!
New York Paris
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
29/57
Locating the messages
New YorkFabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
30/57
Message Density
New York
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
31/57
The New City Landscape Maps
Paris
-
8/6/2019 eResearch Presentation - June '11
32/57
Paris
The New City Landscape Maps
-
8/6/2019 eResearch Presentation - June '11
33/57
New City Landscape
London
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
34/57
-
8/6/2019 eResearch Presentation - June '11
35/57
-
8/6/2019 eResearch Presentation - June '11
36/57
-
8/6/2019 eResearch Presentation - June '11
37/57
New City Landscapes - China
Chongqing BeijingHong Kong
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
38/57
Munich London Paris New York
San FranciscoSydney BarcelonaMoscow
Hong Kong Chungqing Beijing
-
8/6/2019 eResearch Presentation - June '11
39/57
Aspects of time - Tracking
Fabian Neuhaus :- urbantick.blogspot.com/
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
40/57
Recent London Twitter Map
-
8/6/2019 eResearch Presentation - June '11
41/57
Recent London Twitter Map
-
8/6/2019 eResearch Presentation - June '11
42/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
43/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
44/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
45/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
46/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
47/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
48/57
Tweet-o-meter in the Press
-
8/6/2019 eResearch Presentation - June '11
49/57
-
8/6/2019 eResearch Presentation - June '11
50/57
-
8/6/2019 eResearch Presentation - June '11
51/57
-
8/6/2019 eResearch Presentation - June '11
52/57
Main Objectives
- One stop shop for mapping data.
- Mapping your own data or other on-line data
-
Collection of data- Exporting our visualisations to GIS systems
- Making Mapping so easy even a child can do it!
http://gemma.casa.ucl.ac.uk
-
8/6/2019 eResearch Presentation - June '11
53/57
Oliver OBrien :- http://bikes.oobrien.com/http://oliverobrien.co.uk/
London Bike Share Visualisation
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
54/57
London Bike Share Flows
Martin Austwick & Oliver OBrien (@socialphysics, @oobr)
http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238http://casa.ucl.ac.uk/people/person.asp?ID=238 -
8/6/2019 eResearch Presentation - June '11
55/57
bigdatatoolkit.org
-
8/6/2019 eResearch Presentation - June '11
56/57
-
8/6/2019 eResearch Presentation - June '11
57/57
Steven Grayhttp://www.stevenjamesgray.comhttp://www.bigdatatoolkit.orgTwitter: @frogo
http://www.bigdatatoolkit.org/http://www.bigdatatoolkit.org/http://www.stevenjamesgray.com/http://www.stevenjamesgray.com/