postgis gotchasinfo.citusdata.com/rs/235-cne-301/images/postgis_gotchas... · 2015. 11. 24. ·...

PostGIS GotchasThe Things You Didn't Know That Lead to Confusion and Dismay

Paul Ramsey <pramsey@cartodb.com>

Wednesday, November 18, 15

Happy GIS day!What do you callthe day afterGIS day?

PostGIS Dayis tomorrow

Honestly, I have no idea why it works like that...

Projections

Map projectionsare easy to

misunderstand, and is not obvious

how the database handles

I’m a web developer,I don’t have to care aboutprojections.

Latitude, longitude, and leave me alone.

POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

everygeometry

has an“srid”

spatial_ref_sys- srid- srtext

POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

srid = 3005

srid = 3005SELECT proj4text FROM spatial_ref_sys WHERE srid = 3005

srid = 3005

+proj=aea +lat_1=50 +lat_2=58.5 +lat_0=45 +lon_0=-126 +x_0=1000000 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_def

Screwing UpProjections

I’m a web developer,so I loaded the county data in latitude, longitude...

shp2pgsql -‐s 4326 parcels.shp

POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

srid = 4326

ALTER TABLE parcelsALTER COLUMN geomTYPE Geometry(POLYGON, 3005)USING ST_SetSRID(geom, 3005)

POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

srid = 3005

ST_SetSRID(geometry, integer)vs

ST_Transform(geometry, integer)

srid = 3005

ALTER TABLE parcelsALTER COLUMN geomTYPE Geometry(POLYGON, 4326)USING ST_Transform(geom, 4326)

POLYGON((-122.137589562995 49.4494831202169, -122.136067204134 49.4441590695153, -122.137354118353 49.4404247925302, -122.138697187897 49.4395051093253, -122.1451402904 49.436313812165, -122.150704929813 49.434772105515, -122.15191243534 49.4345292778163, -122.156758770153 49.4310075640806, -122.158628595554 49.4297952058489, -122.161356817931 49.4278191153391... ))

srid = 4326

ST_Distance(a,b)

Lack ofProjections

POINT(2.3508 48.8567)Paris

Los AngelesPOINT(-118.25 34.05)

POINT(2.3508 48.8567)Paris

ST_Distance()121.506

Actual Distance: 9107kmWednesday, November 18, 15

POINT(6050592 6877419)Paris

Los AngelesPOINT(1744760 -1146504)

ST_Transform(..., 3857)ST_Distance()

13606727

POINT(6050592 6877419)Paris

Los AngelesPOINT(1744760 -1146504)

ST_Transform(..., 3857)ST_Distance() * cos(radians(lat))

9621409

ST_Distance(a::geography, b::geography) 9107543

Actual Distance: 9107km

POINT(2.3508 48.8567)Paris

Dude, why didn’t you just tell me that right away?

Geography

Pythagoras (Plane)

double dx = x2 - x1double dy = y2 - y1;double d2 = dx * dx + dy * dy;double d = sqrt(d2);

double R = 6371000; /* meters */double d_lat = lat2-lat1; /* radians */double d_lon = lon2-lon1; /* radians */double sin_lat = sin(d_lat/2);double sin_lon = sin(d_lon/2);double a = sin_lat * sin_lat + cos(lat1) * cos(lat2) * sin_lon * sin_lon; double c = 2 * atan2(sqrt(a), sqrt(1-a)); double d = R * c;

Haversine (Sphere)

• ST_Area(g1)

• ST_Distance(g1, g2)

• ST_DWithin(g1, g2, d)

• ST_Intersects(g1, g2)

• ST_Covers(gpoly1, gpt2)

GEOGRAPHY Functions

ST_Buffer(geography)

geometry($1),

ST_Buffer(geography)

ST_Transform( _ST_BestSRID($1) ),

ST_Buffer( $2),

ST_Transform( 4326)

SELECT geography( )

_ST_BestSRID(geog)

_ST_BestSRID(geog1, geog2)

SELECT geography( ST_StartPoint( geometry($1) ))

ST_StartPoint(geography)

GeodeticReasoning

https://trac.osgeo.org/postgis/ticket/3357Wednesday, November 18, 15

ST_Segmentize(geography, 100000)

ST_Segmentize(geometry, 0.01)

Why does ST_Extent(ST_Transform(geom))

not equalST_Transform(ST_Extent(geom))?

ST_Transform()

Dude, stop talking about projections.

UnpredictablePerformance

start-up nerd says....

“I want to reverse-geocode 1000 points per second...”

Given a point, tell me what polygon(s) it is in.

(a) I have a real-time stream of 1000 points per second

(b) I have 1M points I need to process in 1000 seconds

• Lots of small queries

• Find the candidate polygon(s) using r-tree index

• Test each candidate for point-in-polygon edge by edge: O(N)

SELECT * FROM polys WHERE ST_Intersects( geom, ST_SetSRID( ST_MakePoint({x},{y}), {srid}));

• One big query

• Nested loop, tends to send the same polygons into the query, over and over

• PostGIS holds a cache of most recent polygon, with edge index built on it: O(log(N))

SELECT pt.id, poly.idFROM poly, ptWHERE ST_Intersects( poly.geom, pt.geom);

“I want to reverse-geocode 1000 points per second...”

Given a point, tell me what polygon(s) it is in.

Mysterious Operators

Database nerd says,“So you mentioned indexes and r-trees,

but I haven’t seen any operators yet...”

CREATE OR REPLACE FUNCTION ST_Intersects(geometry, geometry)RETURNS booleanAS 'SELECT $1 && $2 AND _ST_Intersects($1,$2)'LANGUAGE 'sql' IMMUTABLE;

Simple FeaturesFor SQL

CREATE OPERATOR CLASS gist_geometry_ops_2d DEFAULT FOR TYPE geometry USING GIST AS STORAGE box2df, OPERATOR 1 << , OPERATOR 2 &< , OPERATOR 3 && , OPERATOR 4 &> , OPERATOR 5 >> , OPERATOR 6 ~= , OPERATOR 7 ~ , OPERATOR 8 @ , OPERATOR 9 &<| , OPERATOR 10 <<| , OPERATOR 11 |>> , OPERATOR 12 |&> , OPERATOR 13 <-> FOR ORDER BY pg_catalog.float_ops, OPERATOR 14 <#> FOR ORDER BY pg_catalog.float_ops, FUNCTION 8 geometry_gist_distance_2d (internal, geometry, int4), FUNCTION 1 geometry_gist_consistent_2d (internal, geometry, int4), FUNCTION 2 geometry_gist_union_2d (bytea, internal), FUNCTION 3 geometry_gist_compress_2d (internal), FUNCTION 4 geometry_gist_decompress_2d (internal), FUNCTION 5 geometry_gist_penalty_2d (internal, internal, internal), FUNCTION 6 geometry_gist_picksplit_2d (internal, internal), FUNCTION 7 geometry_gist_same_2d (geom1 geometry, geom2 geometry, internal);

CREATE OPERATOR CLASS gist_geometry_ops_2d DEFAULT FOR TYPE geometry USING GIST AS ...

CREATE OPERATOR CLASS btree_geometry_ops DEFAULT FOR TYPE geometry USING btree AS OPERATOR 1 < , OPERATOR 2 <= , OPERATOR 3 = , OPERATOR 4 >= , OPERATOR 5 > , FUNCTION 1 geometry_cmp (geom1 geometry, geom2 geometry);

CREATE INDEX myindex ON mytable (geom);

CREATE INDEX myindex USING GIST ON mytable (geom);

SELECT * FROM mytable ORDER BY geom;

• Sort on xmin of bounding box unless they are equal...

• Sort on ymin of bounding box unless they are equal...

• Sort on xmax of bounding box unless they are equal...

• Sort on ymax of bounding box unless they are equal...

• Return “equality”

SELECT * FROM mytable GROUP BY geom;

CREATE INDEX myindex USING GIST ON mytable (geom);

CREATE INDEX myindex USING GIST ON mytable (geom gist_geometry_ops_nd);

CREATE INDEX myindex USING GIST ON mytable (geog);

MysteriousEmptiness

philosophy nerd says“life is emptiness”

ST_Intersection(A, B)

POLYGON EMPTY

is not NULL

ST_Union(C,ST_Intersection(A, B))

ST_Union(C, POLYGON EMPTY)

GEOMETRY EMPTY

• Intersection of disjoint objects

• Simplification using tolerance larger than the object

• Negative buffering using radius larger than the object

• Differencing larger object from smaller

More UnpredictablePerformance

GIS nerd says

“Overlay”

SELECT a.id, b.id, ST_Intersection(a.geom, b.geom)FROM a, bWHERE ST_Intersects(a.geom, b.geom);

SELECT a.id, b.id, CASE WHEN ST_Contains(a.geom, b.geom) THEN b.geom ELSE ST_Intersection(a.geom, b.geom) END AS geomFROM a, bWHERE ST_Intersects(a.geom, b.geom);

Prepared Geometry

•ST_Contains

• ST_ContainsProperly

•ST_Within

•ST_Covers

•ST_Intersects

1 ProblemMany Solutions

philosophy nerd says“when is a buffer

not a buffer?”

SELECT a.id, b.id FROM a, bWHERE ST_Intersects( a.geom, ST_Buffer(b.geom, 500));

SELECT a.id, b.id FROM a, bWHERE ST_Distance(a.geom, b.geom) < 500;

SELECT a.id, b.id FROM a, bWHERE ST_DWithin(a.geom, b.geom, 500);

CREATE OR REPLACE FUNCTION ST_DWithin(geom1 geometry, geom2 geometry, float8) RETURNS boolean AS 'SELECT $1 && ST_Expand($2,$3) AND $2 && ST_Expand($1,$3) AND _ST_DWithin($1, $2, $3)' LANGUAGE 'sql' IMMUTABLE;

RasterDisaster

HiddenFeatures

SET postgis.backend = ‘geos’;SET postgis.backend = ‘sfcgal’;

• ST_3DIntersection

• ST_Tesselate

• ST_3DArea

• ST_Extrude

• ST_ForceLHR

• ST_Orientation

• ST_Minkowski

• ST_StraightSkeleton

New to 2.2!

ST_ApproximateMedialAxis()

CREATE EXTENSION postgis_sfcgal;

CREATE EXTENSION postgis_tiger_geocoder;

CREATE EXTENSION postgis_topology;

CREATE EXTENSION postgis_raster;

Even MoreHidden Features

github.com/pgpointcloud

pgRouting: A Practical Guidehttp://

.com/pgrouting

pgrouting.org

Honestly, I have no idea why it works like that...

postgis gotchasinfo.citusdata.com/rs/235-cne-301/images/postgis_gotchas... · 2015. 11. 24. ·...

Documents

ruby gotchas

java concurrency gotchas

gotchas paintball

concurrency gotchas

postgis 2.1.8 manualpostgis.net/stuff/postgis-2.1.8.pdf ·...

seed funding gotchas

postgis spanish

condor submissions: gotchas

xamarin.android memory management gotchas

hey, what’s new, in postgis two? postgis 1.5

5/4/16 18 postgis topology will replace postgis...

postgis 2.1.10dev...

postgis manual -...

using postgis/postgresql for managing cad and gis data ·...

postgis setup

verilog gotchas part1

postgis topology will replace postgis simplefeature

postgis 2.2.2 manual - open source geospatial...

commiters postgis postgis & qgis - be-opengis-fr ·...

postgresql/postgis: postgis installation & manual use