Apache Spatial Information SystemHiding some of geospatial complexitymartin.desruisseaux@geomatys.com

Apache Spatial Information SystemHiding some of geospatial complexitymartin.desruisseaux@geomatys.com

“We just need latitude and longitude”Simple, isn't it?

“We just need latitude and longitude”Simple, isn't it?

““Referencing by coordinates” in SISReferencing by coordinates” in SIS

Total of about 127,000 lines of code(ignoring about 170,000 lines of comment)

Utilities

Metadata

Referencing

Other

Is Apache SIS a bloated library?Is Apache SIS a bloated library?• Depends on desired accuracy

Library should tell which accuracy to expectLibrary should care about domain of validity

• Depends on desired compliance with authoritiesLibrary should not misuse EPSG identifersLibrary should favour “late-binding” over “early-binding”

• Depends on inter-operability needsWell Known Text (WKT) versions 1 and 2Geographic Markup Language (GML)

Apache SIS referencing module is based on 20 years of experience.SIS is currently strongly driven by implementation of OGC standards.

What OGC gives to Apache SISWhat OGC gives to Apache SIS• Expertise

OGC standards are like blueprintsOGC meetings help to understand the “why” and to anticipate

• InteroperabilityAvoid lock-in to a particular software or vendor

• Tests of correctnessComplete the software-specifc anti-regression tests

Example: “Referencing by coordinates” without guidance from geodesists is rarely done right.Problems are propagated by the port of popular map projections library to various languageswith developers ignoring (conscientiousness or not) fundamental issues.

Various kinds of map projectionsVarious kinds of map projections

Equirectangular Mercator

Mollweide

Stereographic Orthographic

American polyconicLambert conic conformalLambert azimuthal equal area

Images by Strebe - CC BY-SA 3.0

… and many others (> 100), but ● Mercator● Transverse Mercator● Lambert Conic Conformal● Stereographic cover a vast amount of cases.

(only a few are currently implemented in Apache SIS)(only a few are currently implemented in Apache SIS)

Data structuresMap projectionOther transformsOperation inferenceEPSG datasetWKT parsingOther

Map projections are only part of workMap projections are only part of work

Proportion of map projection codein Apache SIS referencing module

Geodetic datumGeodetic datum

Differ in:● Axis length● Eccentricity● Position of ellipsoid centre● Rotation between ellipsoids● And more...

Domainsof validity

Images by NASA / USGS

How many?How many?

Images by Hellerick & al. CC BY-SA 3.0

48 ellipsoids

471 geodetic datums

464 two-dimensional geographiccoordinate reference systems

4406 projectedcoordinate reference systems

In EPSG dataset 8.9.2:In EPSG dataset 8.9.2: Latitudes and longitudeswithout CRS are ambiguous

3 km error in some parts of the world

““I don't need all this complexity”I don't need all this complexity”Let just use the WGS84 datum everywhereLet just use the WGS84 datum everywhere

WGS84 = World Geodetic System 1984This is the datum used by Global Positioning Systems (GPS)

Not exactly what the maps useNot exactly what the maps use

WGS 84(Transit)WGS 84(G730)WGS 84(G873)WGS 84(G1150)WGS 84(G1674)WGS 84(G1762)

Maps in USA and Canada are relativeto the North American Datum (NAD)

● NAD83 is tied to North American Plate● WGS84 is averaged over the world

→ They do not move in the same way● 30 years ago, NAD83 ≈ WGS84● Now, differ by about 1.5 metres

NAD83(86)NAD83(HPGN)NAD83(CORS96)NAD83(2007)NAD83(2011)NAD83(CSRS)

Legacy systems are still in useLegacy systems are still in use• Some boundaries in USA are still legally defned in NAD27• Shift varies for each location, can be close to 80 metres• If the shape is wide enough, transformation causes distortions

• Sometime it is more convenient to work with the original shape

NAD27 NAD83NADCON

Ellipsoidal height odditiesEllipsoidal height oddities

River apparently flowing up(ellipsoidal height increase)

• Difference with geoidal height up to ±100 metres• Global datum is not necessarily the best match

ellipsoid

In summary

If possible, prefer the reference systemdefned by the mapping agency of thecountry where the data are located.

““A simple transformation engine”A simple transformation engine”Can we use WGS84 as the pivot system?Can we use WGS84 as the pivot system?

““Early-binding” approachEarly-binding” approach

World GeodeticSystem 1984

North AmericanDatum 1983

North AmericanDatum 1927

NouvelleTriangulation

Française

RéseauGéodésique

Français 1993

Japanese GeodeticDatum 2000

Japanese GeodeticDatum 2011

World GeodeticSystem 1972 GEOGCS["NTF",

DATUM["Nouvelle Triangulation Française", SPHEROID["Clarke 1880 (IGN)", 6378249.2, 293.466…], TOWGS84TOWGS84[-168, -60, 320][-168, -60, 320]], PRIMEM["Greenwich", 0], UNIT["degree", 0.01745…], AXIS["Latitude", NORTH], AXIS["Longitude", EAST], AUTHORITY["EPSG", "4275"]]

Well Known Text (WKT) format version 1:

““Late-binding” approachLate-binding” approach

World GeodeticSystem 1984

North AmericanDatum 1927

NouvelleTriangulation

Française

RéseauGéodésique

Français 1993

World GeodeticSystem 1972 GEODCRS["NTF",

DATUM["Nouvelle Triangulation Française", ELLIPSOID["Clarke 1880 (IGN)", 6378249.2, 293.466…]], PRIMEM["Greenwich", 0], CS[ellipsoidal, 2], AXIS["Latitude", north], AXIS["Longitude", east], UNIT["degree", 0.01745…], ID["EPSG", 4275]]

Well Known Text (WKT) format version 2:

Japanese GeodeticDatum 2000

Japanese GeodeticDatum 2011

(85)

North AmericanDatum 1983

NA

DC

ON (57)

TOWGS84 is removed in WKT version 2

BOUNDCRS can be usedif a replacement is needed.

What Apache SIS 0.7 can doIntroduction to 9 keys Java methodsWhat Apache SIS 0.7 can doIntroduction to 9 keys Java methods

API noteAPI noteNext slides use SIS static convenience methods.Most of them can be replaced by GeoAPI methods.GeoAPI 3.0 provides implementation independence.

http://www.opengeospatial.org/standards/geoapi/

Use defnitions from a registryUse defnitions from a registry

import org.opengis.referencing.crs.CoordinateReferenceSystem;import org.apache.sis.referencing.CRS;

// Class declaration omitted for brevety

CoordinateReferenceSystem myDataCRS = CRS.forCode(“EPSG:3395”);

// Verify the defnition (WKT 2 syntax)System.out.println(myDataCRS);

1) Search online: http://epsg-registry.org/

2) Give the EPSG code to Apache SIS

3) Verify if the CRS is the expected one

http://spatialreference.org/ and http://epsg.io/are not official sources of EPSG definitions.They differ in axis order and other aspects.

Alternatively, parse GMLAlternatively, parse GML

CoordinateReferenceSystem myDataCRS = CRS.fromXML(“<gml:…>…</gml>”);

<gml:GeodeticCRS gml:id = "epsg-crs-4326"> <gml:identifier codeSpace="IOGP">urn:ogc:def:crs:EPSG::4326</gml:identifier> <gml:name>WGS 84</gml:name> <gml:scope>Horizontal component of 3D system.</gml:scope> <gml:ellipsoidalCS> <gml:EllipsoidalCS gml:id="epsg-cs-6422"> <gml:identifier codeSpace="IOGP">urn:ogc:def:cs:EPSG::6422</gml:identifier> <gml:name>Latitude (north), Longitude (east)</gml:name> <gml:axis> <gml:CoordinateSystemAxis gml:id="epsg-axis-106" uom="urn:ogc:def:uom:EPSG::9122"> <gml:identifier codeSpace="IOGP">urn:ogc:def:axis:EPSG::106</gml:identifier> <gml:name>Geodetic latitude</gml:name> <gml:axisAbbrev>φ </gml:axisAbbrev> <gml:axisDirection codeSpace="EPSG">north</gml:axisDirection> <gml:minimumValue>-90</gml:minimumValue> <gml:maximumValue>+90</gml:maximumValue> <gml:rangeMeaning codeSpace="EPSG">exact</gml:rangeMeaning> </gml:CoordinateSystemAxis> </gml:axis> <gml:axis> <gml:CoordinateSystemAxis gml:id="epsg-axis-107" uom="urn:ogc:def:uom:EPSG::9122"> <gml:identifier codeSpace="IOGP">urn:ogc:def:axis:EPSG::107</gml:identifier> <gml:name>Geodetic longitude</gml:name> <gml:axisAbbrev>λ </gml:axisAbbrev> <gml:axisDirection codeSpace="EPSG">east</gml:axisDirection> <gml:minimumValue>-180</gml:minimumValue> <gml:maximumValue>+180</gml:maximumValue> <gml:rangeMeaning codeSpace="EPSG">wraparound</gml:rangeMeaning> </gml:CoordinateSystemAxis> </gml:axis> </gml:EllipsoidalCS> </gml:ellipsoidalCS>

(continuing...)

<gml:geodeticDatum> <gml:GeodeticDatum gml:id="epsg-datum-6326"> <gml:identifier codeSpace="IOGP">urn:ogc:def:datum:EPSG::6326</gml:identifier> <gml:name>World Geodetic System 1984</gml:name> <gml:scope>Satellite navigation.</gml:scope> <gml:primeMeridian> <gml:PrimeMeridian gml:id = "epsg-meridian-8901"> <gml:identifier codeSpace="IOGP">urn:ogc:def:meridian:EPSG::8901</gml:identifier> <gml:name>Greenwich</gml:name> <gml:greenwichLongitude uom="urn:ogc:def:uom:EPSG::9102">0</gml:greenwichLongitude> </gml:PrimeMeridian> </gml:primeMeridian> <gml:ellipsoid> <gml:Ellipsoid gml:id = "epsg-ellipsoid-7030"> <gml:identifier codeSpace="IOGP">urn:ogc:def:ellipsoid:EPSG::7030</gml:identifier> <gml:name>WGS 84</gml:name> <gml:semiMajorAxis uom="urn:ogc:def:uom:EPSG::9001">6378137.0</gml:semiMajorAxis> <gml:secondDefiningParameter> <gml:SecondDefiningParameter> <gml:inverseFlattening uom="urn:ogc:def:uom:EPSG::9201">298.257223563</gml:inverseFlattening> </gml:SecondDefiningParameter> </gml:secondDefiningParameter> </gml:Ellipsoid> </gml:ellipsoid> </gml:GeodeticDatum> </gml:geodeticDatum></gml:GeodeticCRS>

Alternatively, parse WKTAlternatively, parse WKT

CoordinateReferenceSystem myDataCRS = CRS.fromWKT(“COMPOUNDCRS[…]”);

COMPOUNDCRS["NAD83 UTM 10 + NAVD88", PROJCRS["NAD83 UTM 10", BASEGEODCRS["NAD83", DATUM["North American Datum 1983", ELLIPSOID["GRS 1980", 6378137, 298.257222101]]] CONVERSION["UTM zone 10N", METHOD["Transverse Mercator", ID["EPSG", 9807]], PARAMETER["Longitude of natural origin", -123], PARAMETER["Scale factor", 0.9996], PARAMETER["False easting", 500000]], CS[Cartesian, 2], AXIS["Easting (E)", east], AXIS["Northing (N)", north], LENGTHUNIT["metre", 1]], VERTCRS["NAVD88", VDATUM["North American Vertical Datum 1988"], CS[vertical, 1], AXIS["gravity-related height (H)", up], LENGTHUNIT["metre", 1]]]]

x

y

WKT 1 compatibility issueWKT 1 compatibility issue• Original WKT 1 specifcation (1999) was unclear about some units• Newer WKT 1 specifcation (2001) fxed the ambiguity

• Some popular libraries did not followed

Test: ask the library to show “EPSG:4807” defnition(Note: Paris meridian is at about 2.6 grads ≈ 2.3° from Greenwich)

Wrong: PRIMEM["Paris", 2.33722917], UNIT["grad", 0.015707963267948967]Correct: PRIMEM["Paris", 2.59692130], UNIT["grad", 0.015707963267948967]

● Inconsistent units result in 17 km error at Paris latitude● Same issue with map projection parameters and feet units

WKT migration compatibilityWKT migration compatibility

WKT 1 specifed in:● OGC 99-036● ISO 19125-1:2004

Interpreted by:● GDAL

WKT 1 extended in:● OGC 01-009

Used in:● GeoPackage 1.0● GeoAPI 3.0

WKT 2 specifed in:● OGC 12-063● ISO 19162:2015

Compatible with exceptions✘ TOWGS84 dropped

Compatible✔ unit clarified

Compatibleunder conditions

Apache SIS 0.6+● SIS read automatically● SIS write automatically

● SIS read automatically● SIS write if asked to

● SIS read if asked to● SIS write if asked to

Find coordinate operationFind coordinate operation

import org.opengis.referencing.operation.CoordinateOperation;

// Class declaration omitted for brevety

CoordinateReferenceSystem sourceCRS = // any method shown in previous slidesCoordinateReferenceSystem targetCRS = // any method shown in previous slidesCoordinateOperation op = CRS.findOperation(sourceCRS, targetCRS, null);

// Verify domain of valididty and accuracySystem.out.println("Valid in " + CRS.getGeographicBoundingBox(op));System.out.println("Accuracy " + CRS.getLinearAccuracy(op) + " m");

1) Get two CRS (source and target)

2) Get a coordinate operation from source to target

3) Verify domain of validity and positional accuracy

Example: Apache SIS can recognise that the following are numerically equivalent:

““Find operation” detailedFind operation” detailedStep 1: get EPSG identifers of both CRS● Do not trust the identifer declared in WKT 1 defnition● Search a matching CRS in EPSG dataset even if no identifer is declared● Axis order and units may not match (lot of WKT are non-conform)● Map projection parameters may be declared in different ways

PROJCRS["Scaled Mercator", … CONVERSION["Scaled Mercator", METHOD["Mercator (variant A)"], PARAMETER["Scale factor at natural origin", 0.7558…], PARAMETER["Longitude of natural origin", 100]], …]

PROJCRS["WGS 84 / Mercator 41", … CONVERSION["Mercator 41", METHOD["Mercator (variant B)"], PARAMETER["Latitude of 1st standard parallel", -41], PARAMETER["Longitude of natural origin", 100]], …, ID["EPSG", 3994]]

““Find operation” detailedFind operation” detailed

Work also with three-dimensional geographic CRS(SIS can expand the search to the two-dimensional domain)

Late-bindingStep 2: fnd operation paths specifed by EPSG• Use the EPSG identifers found at step 1• If EPSG dataset declares more than one operation, then:

Choose widest intersection between:– area of interest (specifed by user)– domain of validity (specifed by EPSG)

If same intersection, choose according accuracy• Adjust for axis order and units

““Find operation” detailedFind operation” detailedStep 3: infer operation paths not specifed by EPSG● Separate some CRS components (e.g. temporal)● Change of axis units (e.g. feet to metres)● Change of axis directions (e.g. westing to easting)● Change of axis order (e.g. to right-handed system)● Change of coordinate system type (e.g. polar to Cartesian)● Derived CRS ↔ base CRS conversion (e.g. map projection)● Datum shift using TOWGS84-like information if it exists

Early-binding

“NAD27 to WGS84 (3)” coordinate operation (EPSG:1172) — for Canada

““Find operation” exampleFind operation” example

(x, y, t2)

WGS 84 /World Mercator

TruncatedJulian day

EPSG:3395

(φ, λ, t)

NAD27

Julian day

EPSG:4267

Right-handedNAD27 (λ, φ)

Julian day (t)

NAD27 as 3-D(λ, φ, h1)

Right-handedWGS84 (λ2, φ2)

WGS84 as 3-D(λ2, φ2, h2)

Geocentric(X1, Y1, Z1)

TruncatedJulian day (t2)

Geocentric(X2, Y2, Z2)WGS84

EPSG:4326

Mercator(variant A)

based on

NAD27(φ, λ)

Datum shiftΔX = -10 mΔY = 158 mΔZ = 187 m

EPSG:4267

CoordinateOperation["NAD27 to WGS 84 (3)", SourceCRS[GeodeticCRS["NAD27", Datum["North American Datum 1927", Ellipsoid["Clarke 1866", 6378206.4, 294.9786982138982]], CS[ellipsoidal, 2], Axis["Geodetic latitude (Lat)", north], Axis["Geodetic longitude (Lon)", east], Unit["degree", 0.017453292519943295], Id["EPSG", 4267, "8.9"]], TargetCRS[… definition omitted for brevity …], Method["Geocentric translations (geog2D domain)"], Parameter["X-axis translation", -10, Unit["metre", 1]], Parameter["Y-axis translation", 158, Unit["metre", 1]], Parameter["Z-axis translation", 187, Unit["metre", 1]], OperationAccuracy[20], Scope["Accuracy 15m, 11m and 6m in X, Y and Z axes."], Area["Canada - onshore and offshore."], BBox[40.04, -141.01, 86.46, -47.74], Id["EPSG", 1172, "8.9", URI["urn:ogc:def:coordinateOperation:EPSG:8.9:1172"]], Remark["Derived at 112 stations."]]

… map projection part omitted …

Coordinate OperationCoordinate OperationSystem.out.println(op);

Well Known Text (WKT) version 2

Coordinate operation executionCoordinate operation execution

Concat_MT[ Param_MT["Affine parametric transformation", Parameter["A0", 0.0], Parameter["A1", 1.0], Parameter["B0", 1.0], Parameter["B1", 0.0]], Inverse_MT[Param_MT["Geographic3D to 2D conversion"]], Param_MT["Geographic/geocentric conversions", Parameter["semi_major", 6378206.4, Unit["metre", 1]], Parameter["semi_minor", 6356583.8, Unit["metre", 1]]], Param_MT["Geocentric translations (geocentric domain)", Parameter["X-axis translation", -10.0, Unit["metre", 1]], Parameter["Y-axis translation", 158.0, Unit["metre", 1]], Parameter["Z-axis translation", 187.0, Unit["metre", 1]]], Param_MT["Geocentric_To_Ellipsoid", Parameter["semi_major", 6378137.0, Unit["metre", 1]], Parameter["semi_minor", 6356752.314245179, Unit["metre", 1]]], Param_MT["Geographic3D to 2D conversion"], Param_MT["Mercator (variant A)", Parameter["semi_major", 6378137.0, Unit["metre", 1]], Parameter["semi_minor", 6356752.314245179, Unit["metre", 1]]]]

0 1 01 0 00 0 1

Axis order changefrom (φ, λ) to right-handed

MathTransform mt = op.getMathTransform();System.out.println(mt);

Well Known Text (WKT) version 1

Execution internal Execution internal (what SIS really does)(what SIS really does)

Concat_MT[ Param_MT["Affine parametric transformation", Parameter["A0", 0.0], Parameter["A1", 0.017453292519943295], Parameter["B0", 0.017453292519943295], Parameter["B1", 0.0]], Param_MT["Ellipsoid (radians domain) to centric", Parameter["eccentricity", 0.0822718542230039]], Param_MT["Affine", Parameter["num_row", 4], Parameter["num_col", 4], Parameter["elt_0_0", 1.0000108809202437], Parameter["elt_0_3", -1.567855942887398E-6], Parameter["elt_1_1", 1.0000108809202437], Parameter["elt_1_3", 2.4772123897620888E-5], Parameter["elt_2_2", 1.0000108809202437], Parameter["elt_2_3", 2.931890613199434E-5]], Param_MT["Centric to ellipsoid (radians domain)", Parameter["eccentricity", 0.08181919084262157]], Param_MT["Mercator (radians domain)", Parameter["eccentricity", 0.0818191908426215]], Param_MT["Affine parametric transformation", Parameter["A0", 6378137], Parameter["B1", 6378137]]]

0 1 01 0 00 0 1

1 0 00 1 00 0 h1

0 0 1

to right-handedto 3-D

×

0 π/180 0π/180 0 0

0 0 h1

0 0 1

=

π/180 0 00 π/180 00 0 10 0 1

to radians

×

Pseudo-WKT 1

Transformation outputsTransformation outputs

transform(φ, λ) : xy

∂x/∂φ ∂x/∂λ∂y/∂φ ∂y/∂λ

derivative(φ, λ) :

Example for a two-dimensional map projection:Example for a two-dimensional map projection:

Number of rows or columns depend on the number of dimensions

Transforming pointsTransforming points

import org.opengis.referencing.operation.MathTransform;

// Class declaration omitted for brevety

CoordinateOperation op = CRS.findOperation(sourceCRS, targetCRS, null);MathTransform mt = op.getMathTransform();

double[] sourcePts = {φ1, λ1, φ2, λ2, φ3, λ3, φ4, λ4, …};

double[] targetPts = new double[2 * numPts];mt.transform(sourcePts, 0, targetPts, 0, numPts);

• MathTransform is the type doing the actual workUnmodifable and thread-safeSame transform(…) methods than java.awt.geom.AffineTransform

Transforming envelopesTransforming envelopes• Need smallest box containing the transformed shape

Transforming the 4 corners is not suffcientSIS uses transform derivatives (Jacobian)

import org.opengis.geometry.Envelope;import org.apache.sis.geometry.Envelopes;import org.apache.sis.geometry.Envelope2D;

// Class declaration omitted for brevety

Envelope envelope = new Envelope2D(xmin, ymin, width, height);Envelope transformed = Envelopes.transform(op, envelope);

Transforming envelopesTransforming envelopes• Need smallest box containing the transformed shape

Transforming the 4 corners is not suffcientSIS uses transform derivatives (Jacobian)

Source envelope Transformed shape 4 corners transformation

SummarySummary• Complexity depends on accuracy needs:

● For an accuracy better than 1 kilometre, datum can not be ignored.● For an accuracy of a few metres, local systems may be preferable.

• Some libraries / applications do not know their accuracy.

• Transforming everything to a “pivot” system is not always desirable:• The pivot may be a moving target, while direct and better defned paths exist.• Working with data in their original (legacy) system can avoid geometric distortion.

• Envelope transformations need to take curvature in account.• If still unsure about whether SIS is worth its weight, consider using GeoAPI.• If using WKT 1 format, safer to restrict to metre and degree units

● Unless the library is known to be OGC 01-009 compliant.

Some SIS key static methodsSome SIS key static methods• Get your CoordinateReferenceSystem instances:

• Get your CoordinateOperation instance:

• Is the operation suitable to your needs?

• Transform your data

CoordinateReferenceSystem myDataCRS = CRS.forCode(“EPSG:3395”);

CoordinateReferenceSystem myDataCRS = CRS.fromWKT(“PROJCRS[…]”);

CoordinateOperation op = CRS.findOperation(source, target, aoi);

double positionalAccuracy = CRS.getLinearAccuracy(op);GeographicBoundingBox domain = CRS.getGeographicBoundingBox(op);

Envelope transformed = Envelopes.transform(op, envelope);

Thanks!Thanks!ありがとうありがとうMerci!Merci!

http://www.opengeospatial.org/http://www.geoapi.org/http://sis.apache.org