geographical aspects of big data

Presentatie 2 juni 2014

IM Themadag te GeoFort (Herwijnen) Erik van der Zee (Geodan)

Drs. Erik van der Zee ◦ Physical Geographer and Business Economist

◦ Senior Consultant at Geodan (www.geodan.nl)

◦ PhD Candidate “Added value of location in the Internet of Things and Smart Cities”

◦ Subject matter expert in the Geonovum pilot Making Sense for Society (#labms4s)

E-mail Erik.van.der.Zee@geodan.nl

Twitter @erikvanderzee

What is Big Data?

Spatial Big Data ◦ Big Vector Data (examples)

◦ Big Raster Data (examples)

Big Data Analysis

Spatial Analysis of Big Data ◦ Big Vector Data Analysis (real-time spatial analytics)

◦ Big Raster Data Analysis

Spatial Big Data Visualization (3DJS)

Spatial Big Data Tools and Technologies

Wat is Big Data?

Haseeb Budhani, 2008 “a blanket term for any collection of data sets so large and complex that it becomes difficult to process using on-hand database management tools or traditional data processing applications”

Gartner “high-volume, high-velocity and high-variety information assets that demand cost-effective, innovative forms of information processing for enhanced insight and decision making”

Big Vector Data

2D3D4D datasets (XYZ+Time) “Grote” Geodatasets (BGT/BAG) Lidar Pointclouds ◦ AHN2 (aerial) ◦ Laserscanners op autos (360°)

Real-time Event Streams ◦ Sensors measurements ◦ Actuator Control information ◦ Social Media APIs (twitter, etc.) ◦ “Pasjes en poortjes” (scanners bvb OV checkins)

…

Miljarden laserpunten (terrestrial/aerial) Rotterdam Demo

“Internet of Things”

Real-time sensor and control data (XYZT)

Sensor-Actuator Networks (Nationaal Datawarehouse Wegverkeergegevens)

Real-time data van tienduizenden sensoren elke seconde

Content can be Tekst (Twitter/Whatsapp), Photos (Instagram/Flickr), videos (YouTube), sounds (Soundcloud), …etcetera

Georeferenced Social Media Content

Big Raster Data

Steeds hogere resoluties

Steeds hogere update frequenties (Dynamic HR satellite photos and video)

Steeds meer “banden” (hyperspectral)

Overal land / op water/onder water / in de lucht

Voorbeelden ◦ Luchtfoto’s (5cm resolutie…)

◦ Gigapans

◦ Videobeelden (PTZcams, drones, cars, bodycams)

◦ 360° Panoramas

◦ Multispectral raster data (o.a. satellite imagery)

Real-time Raster data

T=16:00 T=17:00 T=18:00 T=19:00

T=20:00 T=21:00 T=22:00 T=23:00

Hyperspectral data cubes (High Resolution Multispectral Rasters ) = Big Data (petabytes of data)

Hyperspectral data cubes created yearmonthweekday…

Zeer hoge resolutie images die grote gebieden afdekken

http://www.gigapan.com/galleries/11203/gigapans/152220 (demo)

E.g. Google Streetview

Cycomedia Cyclorama/aquarama

DARPA’s big eye: ARGUS-IS 1.8-gigapixel camera for air surveillance ◦ clear images of objects as small as 15 centimeters

from an altitude of six kilometers ◦ One gigapixel is equal to 1,000 megapixels. For

comparison: Modern professional digital cameras have a resolution of about 20 megapixels

Also City Wide Video Surveillance… http://youtu.be/6VkKeM-OK6g?t=8m6s

Past – Present - Future

Types of analysis (Past – Present – Future…) Naald in enorme (BIG) hooiberg vinden (bvb 1 specifiek nummerbord) Soms Trends en patronen destilleren uit berg informatie

Big Vector Data Analysis

(Real-time) analyse van vector data (Event Steam Processing)

Sensing Analysis Act(uat)ing raw events meaningful

events

Data creation

Controlling / alerting /

notification / routing of

objects and people

Data usage

Big Raster Data Analysis

Analyse van rasterdata m.b.v. beeldherkenningsalgoritmen

Meer en meer informatie kan real-time worden afgeleid uit images and videos (creates new derived data that can be analysed in event processing engine) ◦ Face recognition (ov Rotterdam)

◦ Number plates (ANPR)

◦ Traffic signs (Google)

◦ ...

Gezichtsherkenning

Project Ground Truth Google Feature extractie van informatie uit panorama images t.b.v. map Generation and validation

http://youtu.be/FsbLEtS0uls?t=3m52s ◦ Verkeersborden ◦ Rijrichtingen ◦ Bedrijfslogo’s ◦ Straatnamen ◦ Huisnummers ◦ …

Automated Number Plate Recognition (ANPR)

Spatial Big Data Visualization

Trend van visualiseren van BI (big) Data in kaarten (bvb o.b.v. postcode / adres / woonplaats / wijk) naar Ruimtelijke Analyse (spatial analysis) van gegeorefereerde BI Data

Big Data needs New User Interfaces

Nieuwe visualisatie mogelijkheden Voorbeelden http://d3js.org/

Spatial Big Data Tooling

Spatial Extensions voor NoSQL Databases ◦ Neo4J Spatial Extensions (graph database)

◦ MonetDB Spatial Extensions (column store)

◦ Spatial Hadoop http://spatialhadoop.cs.umn.edu/

Spatial Event Stream Processing engines ◦ Esri GeoEvent Processor

◦ Oracle Spatial CEP Engine

◦ Microsoft Stream Insight…

Vragen?