Download - CartoDB Inside Out
CartoDBInside Out
Jorge Sanz · [email protected] · @xurxosanzSASIG · OSGeo.pt · Lisbon · 2015
A technical overview of a web mapping platform
Agenda● Why
● CartoDB nowadays
● Architecture of the platform
○ Components
○ Setup
● How to serve dynamic data at scale
No maps in this talk sorry
https://congosto.cartodb.com/viz/e5da12e2-9fe7-11e4-bc43-0e853d047bba/embed_map
http://elpais.com/elpais/2015/08/20/media/1440070260_752772.html
Why● @jatorre and @saleiva start Vizzuality in April 2008
● Company focused on visualization projects
● Every geospatial project had the same stack:
○ Database + Map Server + APIs + client code
● CartoDB: internal product to deploy faster this geo-stack
Mission
With CartoDB everybody can analyse,visualise and share insights on location data.We are democratising location intelligence
CartoDB nowadays● A VC founded company of ~70 people (NY + Madrid)
● A leading mapping service with +130.000 users
● An Open Source product that can be deployed
on the cloud or on just one server (on-premises)
● Full stack solution from storage to client side libraries
● It’s a mapping editor supported by a platform (APIs)
CartoDB architecture
http://www.di.unito.it/~schifane/smellymaps/explorer.html
https://team.cartodb.com/u/piensaenpixel/viz/8a6530e0-5239-11e5-9cc4-0e0c41326911/public_map
The CartoDB platform● Spatial database for users● NoSQL database for
configurations and cache● Rendering, imports,
database and other services● Cache and Content
Delivery Network● JavaScript Library● Map Editor
Platform
On-premisesor in the Cloud
Data servicesYour data
Editor
The CartoDB platform
CartoDB.js Odyssey.jsTorque.js Leaflet /Gmaps
PostgreSQL
PostGIS
SQL API
Redis
Maps API
TorqueMapnik
Varnish Caching
Fastly CDN
Car
toD
B o
n A
maz
on
el
asti
c R
ecip
es
Car
toD
B o
n
Am
azo
n e
last
ic
Rec
ipes
PostgreSQL
PostGIS
CartoDB Server API
Rest API Imports API
CartoDB Editor
Browser
Cloud setup● 1 - load balancer (nginx)
● 1 - caching (varnish)
● N - UI servers (CartoDB UI)
● N - database servers (user data)
● 1 - Metadata server (CartoDB metadata)
● 1 - redis server (+slave)
● N - Maps API + API SQL servers
Postgres + Postgis● Geodata, users and editor metadata storage
● Tiny Well Known Binary
● CartoDB - Postgres extension
Redis● key-value highly performant storage
● configuration for APIs
● cache
Backend services● Ruby on Rails
● Import API (ogr2ogr)
● Background management tasks
Invalidation service● Node.js + HA Proxy + Consul
● Invalidates caches by request
● Multi-agent, replicated
SQL API● node.js
● Exposes an entry point for running queries
● Acts as an export API
Maps API · Windshaft● node.js + mapnik
● renders tiles from CartoCSS + SQL
● SQL API client
nginx● web server
● load balancer
Varnish + Fastly/GCP● web cache
● Content Delivery Network
CartoDBMap Editor
the easiest web mapping tool
ever
Map Editor● Ruby on Rails + JavaScript
● Consumes the Imports, Maps, SQL API and its own API
● Highly focused on UX-UI
● Manage datasets (schema, privacy, etc)
● Manage maps (basemap, infowindows,
other visual elements, privacy, etc)
● Define layer data source with filters and SQL editor
● Define symbology through wizards and CartoCSS editor
● Publish maps as embeds, links or with CartoDB.js
CartoDB.js● Based on jQuery + Backbone + Underscore
● Leverages Leaflet or Google Maps API
● Easy interaction with the SQL and Maps API
● Manages transparently utf-grids for interactivity
● Allows easy customizations on infowindows,
layer SQL and CartoCSS definitions, etc
Serving data at scale
https://scordivano.cartodb.com/viz/f55a3268-0c29-11e3-827e-5fd91394b353/embed_map
http://mappingkat.github.io/indian-removal/
Story of a tile: first request
Story of a tile: next request
Story of a tile: insert & next request
Cache invalidation
Write queries fire a trigger that updates
cdb_tablemetadata and starts the invalidation
process for the tiles that use that table
Summary● CartoDB is an Open Source, full stack mapping product
● Runs on a single server or distributed
● Delivers to thousands of users the superpowers of Postgis
and Mapnik, not just pretty maps but also new insights
● Provides tools for users and developers
● It’s designed to scale, but we are always improving it
CartoDBInside Out
Jorge Sanz · [email protected] · @xurxosanzSASIG · OSGeo.pt · Lisbon · 2015http://bit.ly/cartodb-sasig2015
Muito obrigado! · Questions?
Links● https://cartodb.com
● https://github.com/CartoDB/cartodb
● https://github.com/CartoDB/cartodb.js
● https://github.com/CartoDB/cartodb-postgresql
● https://github.com/CartoDB/Windshaft-cartodb
● https://github.com/CartoDB/CartoDB-SQL-API
● http://docs.cartodb.com
● http://cartodb.readthedocs.org
