building maps for apps in the cloud - a softlayer use case
DESCRIPTION
Together with Softlayer Snowciety gave a presentation at GOTO Amsterdam 2013 about building custom maps using OpenStreetMap and SRTM data, POstgis/PostgreSQL as a datsbase, Mapnik as a renderer, Tilestance and Apache as the http servers and Leaflet as the javascript client.TRANSCRIPT
![Page 1: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/1.jpg)
![Page 2: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/2.jpg)
![Page 3: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/3.jpg)
![Page 4: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/4.jpg)
![Page 5: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/5.jpg)
![Page 6: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/6.jpg)
Building maps for apps in the cloud
![Page 7: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/7.jpg)
![Page 8: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/8.jpg)
Share your skiing with the world.
Join the Snowciety
![Page 9: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/9.jpg)
Ski tracker
![Page 10: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/10.jpg)
Friend finder
![Page 11: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/11.jpg)
Share your skiing
![Page 12: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/12.jpg)
Google Maps alternativebecause they charge heavy users nowadays
![Page 13: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/13.jpg)
![Page 14: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/14.jpg)
![Page 15: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/15.jpg)
![Page 16: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/16.jpg)
![Page 17: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/17.jpg)
Freedomcreate radical designs
![Page 18: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/18.jpg)
![Page 19: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/19.jpg)
![Page 20: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/20.jpg)
It is all Open Sourceeven the data from NASA
![Page 21: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/21.jpg)
data database renderer http server
![Page 22: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/22.jpg)
client
![Page 23: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/23.jpg)
OpenStreetMapwikipedia for maps
![Page 24: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/24.jpg)
SRTMShuttle Radar Topography Mission
![Page 25: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/25.jpg)
Lots of dataour cluster contains roughly 3,5 terabyte of unrendered data
![Page 26: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/26.jpg)
350 GB of OSMPostgreSQL with PostGIS plugin
![Page 27: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/27.jpg)
3 TB of GEOTIFF
![Page 28: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/28.jpg)
![Page 29: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/29.jpg)
Mapnikrenderer
![Page 30: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/30.jpg)
![Page 31: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/31.jpg)
§
![Page 32: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/32.jpg)
![Page 33: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/33.jpg)
![Page 34: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/34.jpg)
![Page 35: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/35.jpg)
![Page 36: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/36.jpg)
![Page 37: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/37.jpg)
![Page 38: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/38.jpg)
![Page 39: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/39.jpg)
![Page 40: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/40.jpg)
![Page 41: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/41.jpg)
![Page 42: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/42.jpg)
![Page 43: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/43.jpg)
![Page 44: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/44.jpg)
Journey to get here
![Page 45: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/45.jpg)
![Page 46: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/46.jpg)
![Page 47: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/47.jpg)
Journey to get here
![Page 48: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/48.jpg)
§
![Page 49: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/49.jpg)
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
png
256x256
png png png png png png png png png png
![Page 50: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/50.jpg)
Not only server-sidealso client-side library to interact with map
![Page 51: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/51.jpg)
An Open-Source JavaScript Library for Mobile-Friendly Interactive Maps
![Page 52: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/52.jpg)
Apache httpdNo, not really the chopper
![Page 53: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/53.jpg)
Tile StacheTranslating Apache requests into Mapnik render commands
![Page 54: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/54.jpg)
Mapnikthe renderer from before
![Page 55: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/55.jpg)
CachingMemcache, Disk, S3
![Page 56: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/56.jpg)
CachingMemcache, Disk, S3
![Page 57: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/57.jpg)
![Page 58: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/58.jpg)
Importing datatakes a loooooooooooooooooooooooooooooooooooong time
![Page 59: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/59.jpg)
25GBgzipped xml file
![Page 60: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/60.jpg)
and you know how well plain text can be gzipped
file is roughly 250GB unzipped
![Page 61: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/61.jpg)
3 weeks and counting :-|
M1 Large Instance7.5 GiB of memory, 4 EC2 Compute Units
![Page 62: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/62.jpg)
High-Memory Quadruple Extra Large Instance
68.4 GB of memory, 26 EC2 Compute Units, 24 EBS blocks RAID 1$1.640 per Hour = $ 1180 / month
29 hours!
![Page 63: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/63.jpg)
Dedicated hardware68 GB of memory, 8 Cores, 1 SATA disk
$ 700 / month
10 hours
![Page 64: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/64.jpg)
Macbook Pro16 GB of Memory, SSD
![Page 65: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/65.jpg)
8.5 hoursit is all about lots of memory and lots and lots of IO speed
![Page 66: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/66.jpg)
and that was just OSM!
![Page 67: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/67.jpg)
SRTMShuttle Radar Topography Mission
![Page 68: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/68.jpg)
![Page 69: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/69.jpg)
about a minuteper 1” x 1” file
![Page 70: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/70.jpg)
from -180,0 to 180,9028.800 files
![Page 71: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/71.jpg)
20 days laterwe were done importing
![Page 72: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/72.jpg)
![Page 73: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/73.jpg)
So now you knowhow it is made
![Page 74: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/74.jpg)
Future?More sources, better maps, and..
![Page 75: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/75.jpg)
Vector maps!
![Page 76: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/76.jpg)
![Page 77: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/77.jpg)
![Page 78: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/78.jpg)
![Page 79: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/79.jpg)
![Page 80: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/80.jpg)
![Page 81: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/81.jpg)
How do I get started?Mapbox.com
download TileMill
![Page 82: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/82.jpg)
Building your on OSM server (incl all build commands)http://weait.com/content/build-your-own-openstreetmap-server-lucid
Working with terrain data (hillshading, slopeshading, color-relief)http://www.mapbox.com/tilemill/docs/guides/terrain-data/
for when viewing this presentation on Slideshare
Tutorials
![Page 83: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/83.jpg)
THANK YOU@timanrebel
![Page 84: Building maps for apps in the cloud - a Softlayer Use Case](https://reader034.vdocument.in/reader034/viewer/2022051613/54c683064a7959ad128b458b/html5/thumbnails/84.jpg)
Rate me!GOTO Guide App
!