postgis gotchasinfo.citusdata.com/rs/235-cne-301/images/postgis_gotchas... · 2015. 11. 24. ·...

PostGIS GotchasThe Things You Didn't Know That Lead to Confusion and Dismay

Paul Ramsey <[email protected]>

Wednesday, November 18, 15

mailto:[email protected]


Happy GIS day!What do you callthe day afterGIS day?

PostGIS Dayis tomorrow


Honestly, I have no idea why it works like that...


Projections


Map projectionsare easy to

misunderstand, and is not obvious

how the database handles

them


I’m a web developer,I don’t have to care aboutprojections.

Latitude, longitude, and leave me alone.


POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))


everygeometry

has an“srid”

spatial_ref_sys- srid- srtext


POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

srid = 3005


srid = 3005SELECT proj4text FROM spatial_ref_sys WHERE srid = 3005


srid = 3005

+proj=aea +lat_1=50 +lat_2=58.5 +lat_0=45 +lon_0=-126 +x_0=1000000 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_def


Screwing UpProjections


I’m a web developer,so I loaded the county data in latitude, longitude...

shp2pgsql -‐s 4326 parcels.shp


POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

srid = 4326


srid = 4326

ALTER TABLE parcelsALTER COLUMN geomTYPE Geometry(POLYGON, 3005)USING ST_SetSRID(geom, 3005)


POLYGON((1280119.03221134 500217.455871584, 1280261.61218257 499632.634256804, 1280190.99588308 499213.124106239, 1280099.24061359 499105.746442104, 1279651.62405538 498726.116394963, 1279257.64664648 498533.057113174, 1279171.59744916 498501.346411505, 1278841.58918754 498091.419596675, 1278713.36740315 497949.513489055, 1278527.51646549 497719.477195373... ))

srid = 3005


ST_SetSRID(geometry, integer)vs

ST_Transform(geometry, integer)


srid = 3005

ALTER TABLE parcelsALTER COLUMN geomTYPE Geometry(POLYGON, 4326)USING ST_Transform(geom, 4326)


POLYGON((-122.137589562995 49.4494831202169, -122.136067204134 49.4441590695153, -122.137354118353 49.4404247925302, -122.138697187897 49.4395051093253, -122.1451402904 49.436313812165, -122.150704929813 49.434772105515, -122.15191243534 49.4345292778163, -122.156758770153 49.4310075640806, -122.158628595554 49.4297952058489, -122.161356817931 49.4278191153391... ))

srid = 4326


ST_Distance(a,b)


Lack ofProjections


POINT(2.3508 48.8567)Paris

Los AngelesPOINT(-118.25 34.05)


POINT(2.3508 48.8567)Paris


ST_Distance()121.506

Actual Distance: 9107kmWednesday, November 18, 15

POINT(6050592 6877419)Paris

Los AngelesPOINT(1744760 -1146504)

ST_Transform(..., 3857)ST_Distance()

13606727


POINT(6050592 6877419)Paris

Los AngelesPOINT(1744760 -1146504)

ST_Transform(..., 3857)ST_Distance() * cos(radians(lat))

9621409


ST_Distance(a::geography, b::geography) 9107543

Actual Distance: 9107km

POINT(2.3508 48.8567)Paris



Dude, why didn’t you just tell me that right away?


Geography


Pythagoras (Plane)

double dx = x2 - x1double dy = y2 - y1;double d2 = dx * dx + dy * dy;double d = sqrt(d2);


double R = 6371000; /* meters */double d_lat = lat2-lat1; /* radians */double d_lon = lon2-lon1; /* radians */double sin_lat = sin(d_lat/2);double sin_lon = sin(d_lon/2);double a = sin_lat * sin_lat + cos(lat1) * cos(lat2) * sin_lon * sin_lon; double c = 2 * atan2(sqrt(a), sqrt(1-a)); double d = R * c;

Haversine (Sphere)


• ST_Area(g1)

• ST_Distance(g1, g2)

• ST_DWithin(g1, g2, d)

• ST_Intersects(g1, g2)

• ST_Covers(gpoly1, gpt2)

GEOGRAPHY Functions


ST_Buffer(geography)


geometry($1),

ST_Buffer(geography)

ST_Transform( _ST_BestSRID($1) ),

ST_Buffer( $2),

ST_Transform( 4326)

SELECT geography( )


_ST_BestSRID(geog)


_ST_BestSRID(geog)

_ST_BestSRID(geog1, geog2)


SELECT geography( ST_StartPoint( geometry($1) ))

ST_StartPoint(geography)


GeodeticReasoning


https://trac.osgeo.org/postgis/ticket/3357Wednesday, November 18, 15

https://trac.osgeo.org/postgis/ticket/3357

https://trac.osgeo.org/postgis/ticket/3357

https://trac.osgeo.org/postgis/ticket/3357Wednesday, November 18, 15

ST_Segmentize(geography, 100000)


ST_Segmentize(geometry, 0.01)


Why does ST_Extent(ST_Transform(geom))

not equalST_Transform(ST_Extent(geom))?


ST_Transform()


Dude, stop talking about projections.


UnpredictablePerformance


start-up nerd says....


“I want to reverse-geocode 1000 points per second...”

Given a point, tell me what polygon(s) it is in.

(a) I have a real-time stream of 1000 points per second

(b) I have 1M points I need to process in 1000 seconds




• Lots of small queries

• Find the candidate polygon(s) using r-tree index

• Test each candidate for point-in-polygon edge by edge: O(N)

SELECT * FROM polys WHERE ST_Intersects( geom, ST_SetSRID( ST_MakePoint({x},{y}), {srid}));



• One big query

• Nested loop, tends to send the same polygons into the query, over and over

• PostGIS holds a cache of most recent polygon, with edge index built on it: O(log(N))

SELECT pt.id, poly.idFROM poly, ptWHERE ST_Intersects( poly.geom, pt.geom);


“I want to reverse-geocode 1000 points per second...”

Given a point, tell me what polygon(s) it is in.




Mysterious Operators


Database nerd says,“So you mentioned indexes and r-trees,

but I haven’t seen any operators yet...”


CREATE OR REPLACE FUNCTION ST_Intersects(geometry, geometry)RETURNS booleanAS 'SELECT $1 && $2 AND _ST_Intersects($1,$2)'LANGUAGE 'sql' IMMUTABLE;

Simple FeaturesFor SQL


&&


CREATE OPERATOR CLASS gist_geometry_ops_2d DEFAULT FOR TYPE geometry USING GIST AS STORAGE box2df, OPERATOR 1 << , OPERATOR 2 &< , OPERATOR 3 && , OPERATOR 4 &> , OPERATOR 5 >> , OPERATOR 6 ~= , OPERATOR 7 ~ , OPERATOR 8 @ , OPERATOR 9 &<| , OPERATOR 10 <<| , OPERATOR 11 |>> , OPERATOR 12 |&> , OPERATOR 13 <-> FOR ORDER BY pg_catalog.float_ops, OPERATOR 14 <#> FOR ORDER BY pg_catalog.float_ops, FUNCTION 8 geometry_gist_distance_2d (internal, geometry, int4), FUNCTION 1 geometry_gist_consistent_2d (internal, geometry, int4), FUNCTION 2 geometry_gist_union_2d (bytea, internal), FUNCTION 3 geometry_gist_compress_2d (internal), FUNCTION 4 geometry_gist_decompress_2d (internal), FUNCTION 5 geometry_gist_penalty_2d (internal, internal, internal), FUNCTION 6 geometry_gist_picksplit_2d (internal, internal), FUNCTION 7 geometry_gist_same_2d (geom1 geometry, geom2 geometry, internal);


CREATE OPERATOR CLASS gist_geometry_ops_2d DEFAULT FOR TYPE geometry USING GIST AS ...

CREATE OPERATOR CLASS btree_geometry_ops DEFAULT FOR TYPE geometry USING btree AS OPERATOR 1 < , OPERATOR 2 <= , OPERATOR 3 = , OPERATOR 4 >= , OPERATOR 5 > , FUNCTION 1 geometry_cmp (geom1 geometry, geom2 geometry);


CREATE INDEX myindex ON mytable (geom);

CREATE INDEX myindex USING GIST ON mytable (geom);


SELECT * FROM mytable ORDER BY geom;

• Sort on xmin of bounding box unless they are equal...

• Sort on ymin of bounding box unless they are equal...

• Sort on xmax of bounding box unless they are equal...

• Sort on ymax of bounding box unless they are equal...

• Return “equality”

SELECT * FROM mytable GROUP BY geom;


CREATE INDEX myindex USING GIST ON mytable (geom);

CREATE INDEX myindex USING GIST ON mytable (geom gist_geometry_ops_nd);

CREATE INDEX myindex USING GIST ON mytable (geog);

&&&


MysteriousEmptiness


philosophy nerd says“life is emptiness”


A B

ST_Intersection(A, B)


ST_Intersection(A, B)

POLYGON EMPTY

is not NULL


A B

ST_Union(C,ST_Intersection(A, B))

C


ST_Union(C, POLYGON EMPTY)

C


GEOMETRY EMPTY

• Intersection of disjoint objects

• Simplification using tolerance larger than the object

• Negative buffering using radius larger than the object

• Differencing larger object from smaller


More UnpredictablePerformance


GIS nerd says


“Overlay”


SELECT a.id, b.id, ST_Intersection(a.geom, b.geom)FROM a, bWHERE ST_Intersects(a.geom, b.geom);


SELECT a.id, b.id, CASE WHEN ST_Contains(a.geom, b.geom) THEN b.geom ELSE ST_Intersection(a.geom, b.geom) END AS geomFROM a, bWHERE ST_Intersects(a.geom, b.geom);


Prepared Geometry

•ST_Contains

• ST_ContainsProperly

•ST_Within

•ST_Covers

•ST_Intersects


1 ProblemMany Solutions


philosophy nerd says“when is a buffer

not a buffer?”


SELECT a.id, b.id FROM a, bWHERE ST_Intersects( a.geom, ST_Buffer(b.geom, 500));

SELECT a.id, b.id FROM a, bWHERE ST_Distance(a.geom, b.geom) < 500;

SELECT a.id, b.id FROM a, bWHERE ST_DWithin(a.geom, b.geom, 500);


CREATE OR REPLACE FUNCTION ST_DWithin(geom1 geometry, geom2 geometry, float8) RETURNS boolean AS 'SELECT $1 && ST_Expand($2,$3) AND $2 && ST_Expand($1,$3) AND _ST_DWithin($1, $2, $3)' LANGUAGE 'sql' IMMUTABLE;


RasterDisaster


HiddenFeatures


SET postgis.backend = ‘geos’;SET postgis.backend = ‘sfcgal’;


• ST_3DIntersection

• ST_Tesselate

• ST_3DArea

• ST_Extrude

• ST_ForceLHR

• ST_Orientation

• ST_Minkowski

• ST_StraightSkeleton


New to 2.2!

ST_ApproximateMedialAxis()


CREATE EXTENSION postgis_sfcgal;

CREATE EXTENSION postgis_tiger_geocoder;

CREATE EXTENSION postgis_topology;

CREATE EXTENSION postgis_raster;


Even MoreHidden Features


github.com/pgpointcloud


pgRouting: A Practical Guidehttp://

.com/pgrouting

pgrouting.org


Honestly, I have no idea why it works like that...


postgis gotchasinfo.citusdata.com/rs/235-cne-301/images/postgis_gotchas... · 2015. 11. 24. ·...

Documents