location location location
DESCRIPTION
Slides from my ACCU2013 talk on location, GPS, GIS and PostGIS.TRANSCRIPT
LocationLocationLocation
Gavin Heavyside - ACCU 2013 - 12 April [email protected] @gavinheavyside
Friday, 12 April 13
Friday, 12 April 13
GPS, Location, GIS, and Spatial Relationships
• GPS Technology
• GIS and Mapping Concepts
• Spatial DBs and SQL
• Practical Example
Friday, 12 April 13
Friday, 12 April 13
• www.mydrivesolutions.com
• Insurance telematics
• Linux,Ruby, Rails, Postgres, PostGIS, NoSQL, Chef, AWS, C++
Friday, 12 April 13
Friday, 12 April 13
• I have no idea what I’m doing
Friday, 12 April 13
GPS
• Currently 32 satellites
Friday, 12 April 13
GPS
Friday, 12 April 13
Basic GPS Concepts
• 24+ Satellites orbit the earth
• Atomic clocks
• Each constantly transmits its time and location
Friday, 12 April 13
Basic GPS Concepts
• Normally 4 or more sats used by receiver
• Distance to each satellite calculated using speed of light
• Triangulate position
• Derived attributes - speed, bearing
Friday, 12 April 13
http://www.ausairpower.net/TE-GPS-Guided-Weps.html
Friday, 12 April 13
Almanac and Ephemeris
• Almanac: GPS constellation info, orbits
• Ephemeris: orbital and clock correction
• With both in memory, lock is faster
• A-GPS gets ephemeris from internet
Friday, 12 April 13
NMEA 0183
• Serial ASCII data format
• CSV ‘sentences’ for different types of data
Friday, 12 April 13
NMEA Sentences
• GGA - Fix information
• GSA - DOP and active satellites
• GSV - Satellites in view
• RMC - Time, date and position
Friday, 12 April 13
Friday, 12 April 13
Problems with GPS
• Noise
• Urban Canyons
• Multipath signals
• Solar flares and atmospherics
• Mounting of receiver
Friday, 12 April 13
Others
• GLONASS
• BEIDOU/COMPASS
• GALILEO
• IRNSS
Friday, 12 April 13
Smartphone Support
• All major smartphones have GPS sensors
• Balance accuracy, frequency, battery life
• Usually subscribe to service and get called back with location updates
• Alerts: Geofencing, Significant Change
Friday, 12 April 13
HTML5 GeoLocation
Friday, 12 April 13
Friday, 12 April 13
Digital Maps
• Commercial
• NAVTEQ, Tele Atlas
• Free
• OSM
• Online
• Google, Bing, etc
Friday, 12 April 13
Display
• Usually map tiles at different zoom levels
• Server will return tiled images for a bounding box
• Plenty of libraries commercial or free to embed in apps or web pages
• Add your own overlay markers and layers
Friday, 12 April 13
Leaflet
• JS plugin
• Uses OSM or other tile servers
• HW accelerated, touch-compatible
• Markers, Polygons, Layers, GeoJSON
• http://leafletjs.com
Friday, 12 April 13
Friday, 12 April 13
Digital maps for GIS
• Vector representation of your map
• Commercial or OSM
• Use the road or feature geometry, in conjunction with attributes of the features
Friday, 12 April 13
OpenStreetMap
• http://www.openstreetmap.org/
• Community generated and curated
• XML geometry and metadata composed of nodes, ways, and relations
• Can be downloaded in regions, updated incrementally
Friday, 12 April 13
Friday, 12 April 13
Friday, 12 April 13
• Some people when faced with a problem think “I know, I’ll use XML”. Now they have
Friday, 12 April 13
Data Formats
Friday, 12 April 13
Geometry Types
• Points
• Linestrings
• Polygons
• Compound: MultiPoint etc
• 3D: all above with elevation, Polyhedrals
Friday, 12 April 13
Well-Known Text (WKT)
• markup language for vector geometry
• ISO standard, originally by OGC
• Always uses X Y, so watch out when using lat and lon: needs to be (lon lat)
• Also an equivalent Well Known Binary used by databases to store spatial data
• Can embed SRID (see later)
Friday, 12 April 13
WKT Examples
• Point(30 10)
• LINESTRING (30 10, 10 30, 40 40)
• POLYGON ((30 10, 10 20, 20 40, 40 40, 30 10))
• POLYGON ((35 10, 10 20, 15 40,45 45, 35 10),(20 30, 35 35,30 20, 20 30))
Friday, 12 April 13
KML
• XML data format used by Google Earth
• Supports points, lines, markers, polygons, overlays, and much much more
• Zip up with assets (e.g. Collada 3D models) into a .kmz file
Friday, 12 April 13
KML
Friday, 12 April 13
Friday, 12 April 13
GeoJSON
Friday, 12 April 13
ShapeFiles
• Created by ESRI
• Widely used and understood
• Points, Lines, Polygons
• Collection of files: .shp, .shx, .dbf, others
• Can be imported straight into PostGIS with shp2pgsql
Friday, 12 April 13
Friday, 12 April 13
Spatial Reference Systems
• The Earth is not flat
• Neither is it a spheroid
• nor even an ellipsoid
• Spatial Reference Systems approximate the geoid of the earth to an ellipsoid
Friday, 12 April 13
Geoid
Friday, 12 April 13
WGS84
• The most common (and most useful) spatial reference system
• best fitting ellipsoid model to the reference points on the 1996 Earth Gravitational Model geoid
• Used by GPS
Friday, 12 April 13
Friday, 12 April 13
Projections
• Projections
• Cylindrical
• Conic
• Azimuthal
• Orientation
• Oblique
• Equatorial
• Transverse
Friday, 12 April 13
Transverse Mercator
• Some projections look good over most of the globe
• Some are only useful locally (e.g. UTM)
Friday, 12 April 13
Geospatial Databases
• Ordinary RDBs with support for spatial data types
• NoSQL DBs
• MongoDB
• https://github.com/neo4j/spatial
Friday, 12 April 13
PostGIS
• http://postgis.org/
• Geospatial Extensions for PostgreSQL
• Excellent support for spatial data, operations, indexes
• Actively developed
Friday, 12 April 13
Creating a PostGIS DB
Friday, 12 April 13
Spatial Relationships
• Many ways to compare geometries
• Some can compare different types
• Some only valid for same type
• CONTAINS, CROSSES, DISJOINT, DISTANCE, DWITHIN, EQUALS, INTERSECTS, OVERLAPS, TOUCHES, WITHIN
Friday, 12 April 13
ST_Equals
Friday, 12 April 13
ST_Intersects
Friday, 12 April 13
ST_DWithin
Friday, 12 April 13
ST_Transform
Friday, 12 April 13
Geography Data Type
• Usually measurements are in the units of the spatial reference system
• Radians for SRID 4326 (WGS84)
• The geography data type can help
Friday, 12 April 13
Geospatial Indexes
• Indexes on geo columns are necessary
• Covers bounding boxes
• Reduces search space
• Geos used for calculations
Friday, 12 April 13
EXPLAIN
• Use EXPLAIN to check the planner is using spatial indexes, just like other SQL queries
Friday, 12 April 13
Ordnance Survey OpenData
• http://www.ordnancesurvey.co.uk/oswebsite/products/os-opendata.html
• Vector & Raster data
• Maps, Roads, Boundaries, Postcodes
Friday, 12 April 13
Friday, 12 April 13
Code-Point Open
• CSV "Database" of UK postcode
• (Not Northern Ireland)
• 1.6m+ postcodes
• Eastings and Northings for each entry
• Also county/district & NHS info
Friday, 12 April 13
Data Format
• Postcode in field 1
Friday, 12 April 13
Data Format
• Eastings, Northings in fields 3,4
Friday, 12 April 13
OS National Grid
Friday, 12 April 13
Geocoding
• Find the geographic location of a feature, e.g. address, business, POI
• Reverse: Find a feature from a geographic location (e.g. lat/lng)
Friday, 12 April 13
Example App
• Put it all together:
• Open Data
• PostGIS
• CRS transformations
• Geospatial DB queries
Friday, 12 April 13
Friday, 12 April 13
GeoPostCoder
• https://github.com/gavinheavyside/geopostcoder
• Bash and SQL to load postcode geo data
• Simple Ruby web app to query DB
Friday, 12 April 13
Loading Data
• Filter Code-Point Open just to the columns we want
• Load into a temporary table
• Transform National Grid coordinates to WGS Geography data type
Friday, 12 April 13
Friday, 12 April 13
Friday, 12 April 13
Friday, 12 April 13
Friday, 12 April 13
Friday, 12 April 13
Recap
• Lots of sources of location data
• You probably have a GPS receiver with you
• Lots of data available for free
• Plenty of free and open-source tools querying and manipulating spatial data
• Go do something interesting with it :)
Friday, 12 April 13