carto engine
TRANSCRIPT
CARTO ENGINE
JORGE SANZSOLUTIONS ENGINEER & SUPPORT MANAGERGEODEVELOPERS // FEBRUARY 2017
http://bit.ly/170214-carto-api-sdk
WHAT DOES CARTO DO?CARTO’s location intelligence platform helps organizations use and understand the impact and value of place on operational performance, strategic investments, and everyday decisions.
STORE LOCATION ANALYSIS
WORLDWIDE ACCESS
1,200+ CUSTOMERS, 200,000 END-USERS | $30M FUNDING TO DATE | 100+ TEAM MEMBERS
PIONEERS IN LOCATION INTELLIGENCE
https://carto.com/jobs
PRODUCT OFFERING
BUILDER
A web-based drag and drop analysis tool for business users and analysts to discover and predict key insights from location data.
CARTO Builder unleashes the power of location intelligence with self-service, actionable dashboards you can share across your whole organization.
LOCATION INTELLIGENCE FINALLY INTUITIVE
ENGINE
A one-stop shop of geospatial tools, services, and APIs for the rapid development of easy-to-use location applications, CARTO Engine empowers your organization with scalable analysis and enrichment solutions you can fully embed in your web & mobile apps.
● APIs
● SDKs
● Location Data Services
THE MASSIVE POWER BEHIND LOCATION INTELLIGENCE
LOCATION DATA SERVICES
EXCEPTIONAL DATA, TECH & CARTOGRAPHY TO POWER YOUR APPSA set of Location Data Services including vector basemaps, geocoding, routing, and demographic data augmentation services to help extract the full potential of location intelligence.
DATA OBSERVATORYData augmentation services and seamless access to borders, demographics, segmentation and high value location data layers.
The world's most trusted sources of information to pair with your private location data or incorporate into your analysis workflows.
TECHNICALARCHITECTURE
CARTO STACKMOBILE SDK CARTO BUILDER HTML5 USER APP
CARTO.js
Leaflet / Google Maps
Fastly CDN (cloud only)
Varnish caching / nginx web server
Maps API / SQL API / Import API
PostGIS + CARTO-specific functions
PostgreSQL
Data observatory
Location Data Services
CARTO ON AWS /
GOOGLE CLOUD /
ON-PREMISES
3rd PARTY SERVICES
HOW CARTO WORKSPOSTGIS / DATA
SELECT * FROM mytable
CARTOCSS / STYLING#mytable {
marker-type: ellipse;
marker-width: 10;
marker-fill: #5CA2D1;
}
+
MAP TILES
BASEMAP
+
BEAUTIFUL APPSCARTO.js + LEAFLET
GEOSPATIAL DATABASE● PostgreSQL (relational database)
● PostGIS (storage + query of spatial objects for PostgreSQL)
● CARTO extensions
● Exposed through Engine APIs
ETHNIC DIVERSITY OF THE US SHOWING ONE POINT FOR EVERY 100 PEOPLE OF A GIVEN ETHNICITY
API ‘DEEP DIVE’ & EXAMPLES
MAPS APIRetrieve rendered map tiles from data+stylingCreate and manage named maps and templates
SQL APIRun read/write queries against PostgreSQLUse included functions to access Location Data Services and Data Observatory
IMPORT APIImport shp, geojson, csv, kml, gpx…
ENGINE APIs
MANY DIFFERENT FILE FORMATS SUPPORTED
DATA PREPARATIONType guessingAutomatic indexingOverviews generation for large datasets
CSV, SHP, KML, XLS, etc.
SYNC TABLESRefresh remote datasets periodically
ASYNCHRONOUSLarge file support
Upload local or remote files to CARTO
Documentation
IMPORT API: DETAILS
{ "item_queue_id": "efa9925c-31dd-11e4-a95e-0edbca4b5057", "success": true }
curl -v -H "Content-Type: application/json" -d
'{"url":"https://examplehost.com/sample.csv"}'
"https://{account}.cartodb.com/api/v1/imports/?api_key={api_key}"
curl -F file=@/home/documentation/Documents/prism_tour.csv
"https://documentation.cartodb.com/api/v1/imports/?api_key={api_key}"
Create table from a local file
Create table from a remote URL
Response
API DEEP DIVE
IMPORT API: EXAMPLES
THIN WRAPPER AROUND SQL
SYNCHRONOUS/ ASYNCHRONOUSSupport for time consuming “batch” queries
Security constraintsRead/WriteWork like you have your own database
CONNECTORS TO EXTERNAL DATA SOURCES Ogr2ogr, FMECustom
ENRICHED SQLPostGISCARTO’s own functions
Manage and query data
Documentation
SQL API: DETAILS
API DEEP DIVE
SQL API: WORKFLOWS
1. MANAGE YOUR DATA USING SQL
• Example: Create a database table.
• Example: Alter table’s column names or data types.
API DEEP DIVE
SQL API: WORKFLOWS
1. MANAGE YOUR DATA USING SQL
2. DYNAMICALLY RUN GEO-PROCESSING QUERIES, INCLUDING LENGTHY ONES THOUGH BATCH JOBS
• Example: Give me 10 closest coffeeshops to this location.
API DEEP DIVE
SQL API: WORKFLOWS
1. MANAGE YOUR DATA USING SQL
2. DYNAMICALLY RUN GEO-PROCESSING QUERIES, INCLUDING LENGTHY ONES THOUGH BATCH JOBS
3. ACCESS CARTO LOCATION DATA SERVICES (EXAMPLES IN THE NEXT SECTION)
• Example: Give me a walking route from my location to the
nearest coffee shop
API DEEP DIVE
SQL API: WORKFLOWS
1. MANAGE YOUR DATA USING SQL
2. DYNAMICALLY RUN GEO-PROCESSING QUERIES, INCLUDING LENGTHY ONES THOUGH BATCH JOBS
3. ACCESS CARTO LOCATION DATA SERVICES (EXAMPLES IN THE NEXT SECTION)
4. EXPORT DATA
• Example: Export this SQL query as GeoJSON.
{ type: "FeatureCollection", features: [ { type: "Feature", properties: { year: "
2011", month: 10, day: "11", cartodb_id: 1, created_at:
"2012-02-06T22:50:35.778Z", updated_at: "2012-02-12T21:34:08.193Z" }, geometry: {
type: "Point", coordinates: [ -97.335, 35.498 ] } } ] }
http://{account}.cartodb.com/api/v2/sql?format=GeoJSON&q=SELECT * FROM {table}
LIMIT 1&api_key={Your API key}
{ time: 0.006, total_rows: 1, rows: [ { year: " 2011", month: 10, day: "11",
the_geom: "0101000020E610...", cartodb_id: 1, created_at:
"2012-02-06T22:50:35.778Z", updated_at: "2012-02-12T21:34:08.193Z",
the_geom_webmercator: "0101000020110F000..." } ] }
http://{account}.cartodb.com/api/v2/sql?q=SELECT * FROM {table} LIMIT
1&api_key={Your API key}
GET (POST w/payload) request
GET (POST w/payload) request w/ format (e.g., CSV, SHP, SVG, KML, or GeoJSON)
Response
API DEEP DIVE
SQL API: EXAMPLES
Response
HEADERS: 200 OK; application/json
BODY: [{
"job_id": "de305d54-75b4-431b-adb2-eb6b9e546014",
"user": "cartofante"
"query": "UPDATE airports SET type = 'international'",
"status": "pending",
"created_at": "2015-12-15T07:36:25Z",
"updated_at": "2015-12-15T07:36:25Z"
}]
curl -X GET "http://{username}.cartodb.com/api/v2/sql/job/{job_id}"
curl -X POST -H "Content-Type: application/json" -d '{
"query": "CREATE TABLE world_airports AS SELECT a.cartodb_id, a.the_geom,
a.the_geom_webmercator, a.name airport, b.name country FROM world_borders b JOIN
airports a ON ST_Contains(b.the_geom, a.the_geom)"
}' "http://{username}.cartodb.com/api/v2/sql/job"
Create a Batch SQL API job
Get status of a Batch API Job
Response
API DEEP DIVE
SQL API: BATCH EXAMPLES
SELECT * FROM {table} ORDER BY the_geom <-> CDB_LatLng(42.672291,-71.226982)
LIMIT 65
SELECT *, ST_Distance(the_geom, CDB_LatLng(42.672291,-71.226982)) d FROM {table}
SELECT cartodb_id, the_geom, the_geom_webmercator FROM {table}Select CARTO’s specific fields
SELECT ST_MakeLine(the_geom_webmercator) as the_geom_webmercator FROM {table}Join all points in table to make a line
API DEEP DIVE
SQL API: ANALYSIS (1)
Create a virtual column with the distance to a given point
Order by distance to a given point and limit the number of results
SELECT ST_Area(the_geom::geography) as area FROM {table}
UPDATE {table_1} SET points_in = (SELECT count(*) FROM {table_2} WHERE
ST_Intersects(the_geom, {table_1}.the_geom))
Create a new column in table 1 with the number of points from table 2 that fall inside each of table 1’s polygons
SELECT count(*), {table_1}.the_geom_webmercator FROM {table_1}, {table_2} WHERE
ST_Intersects({table_1}.the_geom, {table_2}.the_geom)
Now add the same column dynamically, with the_geom_webmercator
API DEEP DIVE
SQL API: ANALYSIS (2)
Create a virtual column with the area of each polygon in a table
Learn more at http://postgis.net/docs/reference.html.
API DEEP DIVE
DATA SERVICES API: WORKFLOWS
Documentation
1. GEOCODE POINTS BY STREET ADDRESS OR LOCATION NAMES USING SQL API
API DEEP DIVE
DATA SERVICES API: WORKFLOWS
1. GEOCODE POINTS BY STREET ADDRESS OR LOCATION NAMES USING SQL API
2. OBTAIN ISOLINES FOR A POINT USING SQL API
API DEEP DIVE
DATA SERVICES API: WORKFLOWS
1. GEOCODE POINTS BY STREET ADDRESS OR LOCATION NAMES USING SQL API
2. OBTAIN ISOLINES FOR A POINT USING SQL API
3. OBTAIN POINT-TO-POINT ROUTING USING SQL API
API DEEP DIVE
DATA SERVICES API: WORKFLOWS
1. GEOCODE POINTS BY STREET ADDRESS OR LOCATION NAMES USING SQL API
2. OBTAIN ISOLINES FOR A POINT USING SQL API
3. OBTAIN POINT-TO-POINT ROUTING USING SQL API
4. OBTAIN DEMOGRAPHICS AND OTHER DATA OBSERVATORY MEASURES AND BOUNDARIES USING SQL API
• Example: Give me the duration, distance, and geometry of a
route between these two points.
• Example: Give me a demographic snapshot of this location.
UPDATE {tablename}
SET local_male_population = OBS_GetUSCensusMeasure(the_geom, 'Male Population')
INSERT INTO {table} (the_geom) SELECT the_geom FROM cdb_isochrone('POINT(-3.70568
40.42028)'::geometry, 'car', ARRAY[300, 900, 12000]::integer[],
ARRAY['mode_traffic=enabled','quality=3']::text[])
UPDATE {tablename} SET the_geom = cdb_geocode_admin0_polygon({country_column})Geocode a table with country names
Insert an Isochrone geometry into a table
API DEEP DIVE
DATA SERVICES API: EXAMPLES
INSERT INTO <TABLE> (duration, length, the_geom) SELECT duration, length, shape
FROM cdb_route_point_to_point(
'POINT(-3.70237112 40.41706163)'::geometry,
'POINT(-3.69909883 40.41236875)'::geometry, 'car')
Insert an Route details into a table
Update a table with a Data Observatory Measure
MAPS API: DETAILS● Map tile rendering from SQL + CartoCSS
● Layer blending - layer and sublayer support
● Server side: support large amounts of features
● Client side: animated + aggregated Torque maps
● Named maps - maps out of private datasets
Documentation
API DEEP DIVE
MAPS API: WORKFLOWS
1. CREATE MAPS BASED ON YOUR DATA AND DISPLAY THEM ON LEAFLET/GOOGLE MAPS OR ANY OTHER CLIENT
• Anonymous maps:
• Access public data with read-only SQL and CartoCSS.
• Create maps from Javascript apps (no authentication).
• Named maps:
• Access private data by setting and using server-side SQL
and CartoCSS templates.
• Requires authentication
API DEEP DIVE
MAPS API: WORKFLOWS
1. CREATE MAPS BASED ON YOUR DATA AND DISPLAY THEM ON LEAFLET/GOOGLE MAPS OR ANY OTHER CLIENT
2. USE THE STATIC MAPS TO GENERATE IMAGES
• Example: Give me a map with this data from this style.
var mapConfig = { "version": "1.0.1", "layers": [{ "type": "cartodb",
"options": { "cartocss_version": "2.1.1", "cartocss": "#layer {
polygon-fill: #FFF; }", "sql": "select * from table_name" } }] };
$.ajax({ crossOrigin: true,
type: 'POST',
dataType: 'json',
contentType: 'application/json',
url: 'http://{account}.cartodb.com/api/v1/map',
data: JSON.stringify(mapConfig),
success: function(data) {
var templateUrl = 'http://{account}.cartodb.com/api/v1/map/' +
data.layergroupid + ‘/{z}/{x}/{y}.png';
console.log(templateUrl);
}
});
http://{account}.cartodb.com/api/v1/map/293ebfc4a757de0277336a6e2ab4bb15:0/{
z}/{x}/{y}.png
Get template URL for an Anonymous Map (public)
API DEEP DIVE
MAPS API: EXAMPLES
mapconfig.json: { "version": "0.0.1", "name": "test", "auth": { "method": "open" },
"layergroup": { "layers": [{ "type": "cartodb", "options": { "cartocss_version": "2.1.1",
"cartocss": "#layer { polygon-fill: #FFF; }", "sql": "select * table_name" } }] } }
curl 'https://{account}.cartodb.com/api/v1/map/?api_key=APIKEY' \
-H 'Content-Type: application/json' -d @mapconfig.json
curl -X POST 'http://{account}.cartodb.com/api/v1/map/named/:template_id' \
-H 'Content-Type: application/json'
{ "cdn_url": { "http": "ashbu.cartocdn.com", "https": "cartocdn-ashbu.global.ssl.fastly.net"
}, "layergroupid": "c01a54877c62831bb51720263f91fb33:0", "last_updated":
"1970-01-01T00:00:00.000Z" }
http://{account}.cartodb.com/api/v1/map/c01a54877c62831bb51720263f91fb33:0/{z}/{x}/{y}.png
Get template URL for a Named Map (auth required)
API DEEP DIVE
MAPS API: EXAMPLES
https://{username}.cartodb.com/api/v1/map/named/{named_map_name}/{layer}/{z}/{x}/{y}.png
https://example.cartodb.com/api/v1/map/named/my_map/all/0/0/0.png
Get X/Y/Z direct tiles URL for a Named Map
API DEEP DIVE
MAPS API: EXAMPLES
BACKEND SDKsImport API / SQL API wrappers for Python, node.js and others
At this moment only Python SDK is actively maintained
Javascript SDK for HTML5 app development.
CARTO.js: DETAILS
● Based on jQuery + Backbone + Underscore
● Interfaces with SQL + Maps APIs to make client-side development easy
● Built on Leaflet - leverage existing knowledge and an open source community
● Integrates into Google Maps natively, can be used from OpenLayers, and others
Documentación
MOBILE SDK
Custom mobile maps SDK for Android, iOS, WP10 and Xamarin supporting online and offline solutions.
Benefit from the performance and scalability of the CARTO SDK and enable online & offline base maps, online & offline routing, GIS editing, 3D features and indoor mapping for your apps.
https://github.com/CartoDB/mobile-sdk https://github.com/CartoDB/mobile-android-samples….
● APIKeyAuthClient
● FileImportJob
● SQLClient
● DatasetManager
● VisualizationManager
Python SDK
https://github.com/CartoDB/carto-python/tree/1.0.0
Ejemplos
OTHER RESOURCES
● Main documentation site● CARTO Workshop● CARTO guides● CARTO team map● Antipodes Map (OpenLayers + CARTO) · repo● Cesium examples · repo
