cartodb inside out

Click here to load reader

Post on 14-Apr-2017

1.234 views

Category:

Internet

0 download

Embed Size (px)

TRANSCRIPT

  • CartoDBInside Out

    Jorge Sanz [email protected] @xurxosanzSASIG OSGeo.pt Lisbon 2015

    A technical overview of a web mapping platform

  • Agenda Why

    CartoDB nowadays

    Architecture of the platform

    Components

    Setup

    How to serve dynamic data at scale

    No maps in this talk sorry

  • https://congosto.cartodb.com/viz/e5da12e2-9fe7-11e4-bc43-0e853d047bba/embed_map

    https://congosto.cartodb.com/viz/e5da12e2-9fe7-11e4-bc43-0e853d047bba/embed_maphttps://congosto.cartodb.com/viz/e5da12e2-9fe7-11e4-bc43-0e853d047bba/embed_map

  • http://elpais.com/elpais/2015/08/20/media/1440070260_752772.html

    http://elpais.com/elpais/2015/08/20/media/1440070260_752772.htmlhttp://elpais.com/elpais/2015/08/20/media/1440070260_752772.html

  • Why @jatorre and @saleiva start Vizzuality in April 2008

    Company focused on visualization projects

    Every geospatial project had the same stack:

    Database + Map Server + APIs + client code

    CartoDB: internal product to deploy faster this geo-stack

  • Mission

    With CartoDB everybody can analyse,visualise and share insights on location data.We are democratising location intelligence

  • CartoDB nowadays A VC founded company of ~70 people (NY + Madrid)

    A leading mapping service with +130.000 users

    An Open Source product that can be deployed

    on the cloud or on just one server (on-premises)

    Full stack solution from storage to client side libraries

    Its a mapping editor supported by a platform (APIs)

  • CartoDB architecture

  • http://www.di.unito.it/~schifane/smellymaps/explorer.html

    http://www.di.unito.it/~schifane/smellymaps/explorer.htmlhttp://www.di.unito.it/~schifane/smellymaps/explorer.html

  • https://team.cartodb.com/u/piensaenpixel/viz/8a6530e0-5239-11e5-9cc4-0e0c41326911/public_map

    https://team.cartodb.com/u/piensaenpixel/viz/8a6530e0-5239-11e5-9cc4-0e0c41326911/public_maphttps://team.cartodb.com/u/piensaenpixel/viz/8a6530e0-5239-11e5-9cc4-0e0c41326911/public_map

  • The CartoDB platform

    Spatial database for users NoSQL database for

    configurations and cache Rendering, imports,

    database and other services Cache and Content

    Delivery Network JavaScript Library Map Editor

    Platform

    On-premisesor in the Cloud

    Data servicesYour data

    Editor

  • The CartoDB platform

    CartoDB.js Odyssey.jsTorque.js Leaflet /Gmaps

    PostgreSQL

    PostGIS

    SQL API

    Redis

    Maps API

    TorqueMapnik

    Varnish Caching

    Fastly CDN

    Car

    toD

    B o

    n A

    maz

    on

    el

    asti

    c R

    ecip

    es

    Car

    toD

    B o

    n

    Am

    azo

    n e

    last

    ic

    Rec

    ipes

    PostgreSQL

    PostGIS

    CartoDB Server API

    Rest API Imports API

    CartoDB Editor

    Browser

  • Cloud setup 1 - load balancer (nginx)

    1 - caching (varnish)

    N - UI servers (CartoDB UI)

    N - database servers (user data)

    1 - Metadata server (CartoDB metadata)

    1 - redis server (+slave)

    N - Maps API + API SQL servers

  • Postgres + Postgis Geodata, users and editor metadata storage

    Tiny Well Known Binary

    CartoDB - Postgres extension

    Redis key-value highly performant storage

    configuration for APIs

    cache

  • Backend services Ruby on Rails

    Import API (ogr2ogr)

    Background management tasks

    Invalidation service Node.js + HA Proxy + Consul

    Invalidates caches by request

    Multi-agent, replicated

  • SQL API node.js

    Exposes an entry point for running queries

    Acts as an export API

    Maps API Windshaft node.js + mapnik

    renders tiles from CartoCSS + SQL

    SQL API client

  • nginx web server

    load balancer

    Varnish + Fastly/GCP web cache

    Content Delivery Network

  • CartoDBMap Editor

    the easiest web mapping tool

    ever

  • Map Editor Ruby on Rails + JavaScript

    Consumes the Imports, Maps, SQL API and its own API

    Highly focused on UX-UI

    Manage datasets (schema, privacy, etc)

    Manage maps (basemap, infowindows,

    other visual elements, privacy, etc)

    Define layer data source with filters and SQL editor

    Define symbology through wizards and CartoCSS editor

    Publish maps as embeds, links or with CartoDB.js

  • CartoDB.js Based on jQuery + Backbone + Underscore

    Leverages Leaflet or Google Maps API

    Easy interaction with the SQL and Maps API

    Manages transparently utf-grids for interactivity

    Allows easy customizations on infowindows,

    layer SQL and CartoCSS definitions, etc

  • Serving data at scale

  • https://scordivano.cartodb.com/viz/f55a3268-0c29-11e3-827e-5fd91394b353/embed_map

    https://scordivano.cartodb.com/viz/f55a3268-0c29-11e3-827e-5fd91394b353/embed_maphttps://scordivano.cartodb.com/viz/f55a3268-0c29-11e3-827e-5fd91394b353/embed_map

  • http://mappingkat.github.io/indian-removal/

  • Story of a tile: first request

  • Story of a tile: next request

  • Story of a tile: insert & next request

  • Cache invalidation

    Write queries fire a trigger that updates

    cdb_tablemetadata and starts the invalidation

    process for the tiles that use that table

  • Summary CartoDB is an Open Source, full stack mapping product

    Runs on a single server or distributed

    Delivers to thousands of users the superpowers of Postgis

    and Mapnik, not just pretty maps but also new insights

    Provides tools for users and developers

    Its designed to scale, but we are always improving it

  • We are hiring!!

    http://cartodb.com/jobs

    http://cartodb.com/jobshttp://cartodb.com/jobs

  • CartoDBInside Out

    Jorge Sanz [email protected] @xurxosanzSASIG OSGeo.pt Lisbon 2015http://bit.ly/cartodb-sasig2015

    Muito obrigado! Questions?

    http://bit.ly/cartodb-sasig2015http://bit.ly/cartodb-sasig2015

  • Links https://cartodb.com

    https://github.com/CartoDB/cartodb

    https://github.com/CartoDB/cartodb.js

    https://github.com/CartoDB/cartodb-postgresql

    https://github.com/CartoDB/Windshaft-cartodb

    https://github.com/CartoDB/CartoDB-SQL-API

    http://docs.cartodb.com

    http://cartodb.readthedocs.org

    https://cartodb.comhttps://cartodb.comhttps://github.com/CartoDB/cartodbhttps://github.com/CartoDB/cartodbhttps://github.com/CartoDB/cartodb.jshttps://github.com/CartoDB/cartodb.jshttps://github.com/CartoDB/cartodb-postgresqlhttps://github.com/CartoDB/cartodb-postgresqlhttps://github.com/CartoDB/Windshaft-cartodbhttps://github.com/CartoDB/Windshaft-cartodbhttps://github.com/CartoDB/CartoDB-SQL-APIhttps://github.com/CartoDB/CartoDB-SQL-APIhttp://docs.cartodb.com/http://docs.cartodb.com/http://cartodb.readthedocs.orghttp://cartodb.readthedocs.org